WALProcedureStore (Apache HBase 2.5.0 API)

java.lang.Object
- org.apache.hadoop.hbase.procedure2.store.ProcedureStoreBase
- - org.apache.hadoop.hbase.procedure2.store.wal.WALProcedureStore

All Implemented Interfaces:

ProcedureStore

Deprecated.
Since 2.3.0, will be removed in 4.0.0. Keep here only for rolling upgrading, now we use the new region based procedure store.
```
@Deprecated
 @InterfaceAudience.Private
public class WALProcedureStore
extends ProcedureStoreBase
```
WAL implementation of the ProcedureStore.
When starting, the upper layer will first call start(int), then recoverLease(), then ProcedureStore.load(ProcedureLoader).
In recoverLease(), we will get the lease by closing all the existing wal files(by calling recoverFileLease), and creating a new wal writer. And we will also get the list of all the old wal files.
FIXME: notice that the current recover lease implementation is problematic, it can not deal with the races if there are two master both wants to acquire the lease...
In ProcedureStore.load(ProcedureLoader) method, we will load all the active procedures. See the comments of this method for more details.
The actual logging way is a bit like our FileSystem based WAL implementation as RS side. There is a slots, which is more like the ring buffer, and in the insert, update and delete methods we will put thing into the slots and wait. And there is a background sync thread(see the syncLoop() method) which get data from the slots and write them to the FileSystem, and notify the caller that we have finished.
TODO: try using disruptor to increase performance and simplify the logic?
The storeTracker keeps track of the modified procedures in the newest wal file, which is also the one being written currently. And the deleted bits in it are for all the procedures, not only the ones in the newest wal file. And when rolling a log, we will first store it in the trailer of the current wal file, and then reset its modified bits, so that it can start to track the modified procedures for the new wal file.
The holdingCleanupTracker is used to test whether we are safe to delete the oldest wal file. When there are log rolling and there are more than 1 wal files, we will make use of it. It will first be initialized to the oldest file's tracker(which is stored in the trailer), using the method ProcedureStoreTracker.resetTo(ProcedureStoreTracker, boolean), and then merge it with the tracker of every newer wal files, using the ProcedureStoreTracker.setDeletedIfModifiedInBoth(ProcedureStoreTracker). If we find out that all the modified procedures for the oldest wal file are modified or deleted in newer wal files, then we can delete it. This is because that, every time we call ProcedureStore.insert(Procedure[]) or ProcedureStore.update(Procedure), we will persist the full state of a Procedure, so the earlier wal records for this procedure can all be deleted.

See Also:

for printing content of a single WAL., to parse a directory of MasterWALProcs.

Nested Class Summary

Nested Classes
Modifier and Type Class and Description

private static class WALProcedureStore.PushType
Deprecated.

static class WALProcedureStore.SyncMetrics
Deprecated.
- Nested classes/interfaces inherited from interface org.apache.hadoop.hbase.procedure2.store.ProcedureStore
  ProcedureStore.ProcedureIterator, ProcedureStore.ProcedureLoader, ProcedureStore.ProcedureStoreListener

Nested Classes
Modifier and Type	Class and Description
`private static class`	`WALProcedureStore.PushType` Deprecated.
`static class`	`WALProcedureStore.SyncMetrics` Deprecated.

Field Summary

Fields
Modifier and Type	Field and Description
`private org.apache.hadoop.conf.Configuration`	`conf` Deprecated.
`private Set<ProcedureWALFile>`	`corruptedLogs` Deprecated.
`private static boolean`	`DEFAULT_EXEC_WAL_CLEANUP_ON_LOAD_CONF_KEY` Deprecated.
`private static int`	`DEFAULT_MAX_RETRIES_BEFORE_ROLL` Deprecated.
`private static int`	`DEFAULT_MAX_SYNC_FAILURE_ROLL` Deprecated.
`private static int`	`DEFAULT_PERIODIC_ROLL` Deprecated.
`private static int`	`DEFAULT_ROLL_RETRIES` Deprecated.
`private static long`	`DEFAULT_ROLL_THRESHOLD` Deprecated.
`private static int`	`DEFAULT_SYNC_STATS_COUNT` Deprecated.
`private static int`	`DEFAULT_SYNC_WAIT_MSEC` Deprecated.
`private static boolean`	`DEFAULT_USE_HSYNC` Deprecated.
`private static int`	`DEFAULT_WAIT_BEFORE_ROLL` Deprecated.
`private static int`	`DEFAULT_WAL_COUNT_WARN_THRESHOLD` Deprecated.
`private boolean`	`enforceStreamCapability` Deprecated.
`static String`	`EXEC_WAL_CLEANUP_ON_LOAD_CONF_KEY` Deprecated.
`private static Comparator<org.apache.hadoop.fs.FileStatus>`	`FILE_STATUS_ID_COMPARATOR` Deprecated.
`private long`	`flushLogId` Deprecated.
`private org.apache.hadoop.fs.FileSystem`	`fs` Deprecated.
`private ProcedureStoreTracker`	`holdingCleanupTracker` Deprecated.
`private AtomicBoolean`	`inSync` Deprecated.
`private AtomicLong`	`lastRollTs` Deprecated.
`private LeaseRecovery`	`leaseRecovery` Deprecated.
`private AtomicBoolean`	`loading` Deprecated.
`private ReentrantLock`	`lock` Deprecated.
`private static org.slf4j.Logger`	`LOG` Deprecated.
`static String`	`LOG_PREFIX` Deprecated.
`private LinkedList<ProcedureWALFile>`	`logs` Deprecated.
`static String`	`MASTER_PROCEDURE_LOGDIR` Deprecated. Used to construct the name of the log directory for master procedures
`static String`	`MAX_RETRIES_BEFORE_ROLL_CONF_KEY` Deprecated.
`static String`	`MAX_SYNC_FAILURE_ROLL_CONF_KEY` Deprecated.
`private int`	`maxRetriesBeforeRoll` Deprecated.
`private int`	`maxSyncFailureRoll` Deprecated.
`static String`	`PERIODIC_ROLL_CONF_KEY` Deprecated.
`private int`	`periodicRollMsec` Deprecated.
`static String`	`ROLL_RETRIES_CONF_KEY` Deprecated.
`static String`	`ROLL_THRESHOLD_CONF_KEY` Deprecated.
`private int`	`rollRetries` Deprecated.
`private long`	`rollThreshold` Deprecated.
`private int`	`runningProcCount` Deprecated.
`private Condition`	`slotCond` Deprecated.
`private int`	`slotIndex` Deprecated.
`private ByteSlot[]`	`slots` Deprecated.
`private LinkedTransferQueue<ByteSlot>`	`slotsCache` Deprecated.
`static String`	`STORE_WAL_SYNC_STATS_COUNT` Deprecated.
`private ProcedureStoreTracker`	`storeTracker` Deprecated.
`private org.apache.hadoop.fs.FSDataOutputStream`	`stream` Deprecated.
`static String`	`SYNC_WAIT_MSEC_CONF_KEY` Deprecated.
`private Condition`	`syncCond` Deprecated.
`private AtomicReference<Throwable>`	`syncException` Deprecated.
`private AtomicLong`	`syncId` Deprecated.
`private int`	`syncMaxSlot` Deprecated.
`private org.apache.hbase.thirdparty.org.apache.commons.collections4.queue.CircularFifoQueue<WALProcedureStore.SyncMetrics>`	`syncMetricsQueue` Deprecated.
`private Thread`	`syncThread` Deprecated.
`private int`	`syncWaitMsec` Deprecated.
`private AtomicLong`	`totalSynced` Deprecated.
`static String`	`USE_HSYNC_CONF_KEY` Deprecated.
`private boolean`	`useHsync` Deprecated.
`static String`	`WAIT_BEFORE_ROLL_CONF_KEY` Deprecated.
`private int`	`waitBeforeRoll` Deprecated.
`private Condition`	`waitCond` Deprecated.
`static String`	`WAL_COUNT_WARN_THRESHOLD_CONF_KEY` Deprecated.
`private org.apache.hadoop.fs.Path`	`walArchiveDir` Deprecated.
`private int`	`walCountWarnThreshold` Deprecated.
`private org.apache.hadoop.fs.Path`	`walDir` Deprecated.
`private static org.apache.hadoop.fs.PathFilter`	`WALS_PATH_FILTER` Deprecated.

Constructor Summary

Constructors
Constructor and Description
`WALProcedureStore(org.apache.hadoop.conf.Configuration conf, LeaseRecovery leaseRecovery)` Deprecated.
`WALProcedureStore(org.apache.hadoop.conf.Configuration conf, org.apache.hadoop.fs.Path walDir, org.apache.hadoop.fs.Path walArchiveDir, LeaseRecovery leaseRecovery)` Deprecated.

Method Summary

All Methods Static Methods Instance Methods Concrete Methods Deprecated Methods
Modifier and Type	Method and Description
`private ByteSlot`	`acquireSlot()` Deprecated.
`private void`	`buildHoldingCleanupTracker()` Deprecated.
`private void`	`closeCurrentLogStream(boolean abort)` Deprecated.
`void`	`delete(long procId)` Deprecated. The specified procId was removed from the executor, due to completion, abort or failure.
`private void`	`delete(long[] procIds)` Deprecated.
`void`	`delete(long[] procIds, int offset, int count)` Deprecated. The specified procIds were removed from the executor, due to completion, abort or failure.
`void`	`delete(Procedure<?> proc, long[] subProcIds)` Deprecated. The parent procedure completed.
`ArrayList<ProcedureWALFile>`	`getActiveLogs()` Deprecated.
`Set<ProcedureWALFile>`	`getCorruptedLogs()` Deprecated.
`org.apache.hadoop.fs.FileSystem`	`getFileSystem()` Deprecated.
`protected org.apache.hadoop.fs.Path`	`getLogFilePath(long logId)` Deprecated.
`private org.apache.hadoop.fs.FileStatus[]`	`getLogFiles()` Deprecated.
`private static long`	`getLogIdFromName(String name)` Deprecated.
`private static long`	`getMaxLogId(org.apache.hadoop.fs.FileStatus[] logFiles)` Deprecated. Make sure that the file set are gotten by calling `getLogFiles()`, where we will sort the file set by log id.
`long`	`getMillisFromLastRoll()` Deprecated.
`long`	`getMillisToNextPeriodicRoll()` Deprecated.
`int`	`getNumThreads()` Deprecated. Returns the number of threads/slots passed to start()
`ProcedureStoreTracker`	`getStoreTracker()` Deprecated.
`ArrayList<WALProcedureStore.SyncMetrics>`	`getSyncMetrics()` Deprecated.
`(package private) org.apache.hadoop.fs.Path`	`getWalArchiveDir()` Deprecated.
`org.apache.hadoop.fs.Path`	`getWALDir()` Deprecated.
`private ProcedureWALFile`	`initOldLog(org.apache.hadoop.fs.FileStatus logFile, org.apache.hadoop.fs.Path walArchiveDir)` Deprecated. Loads given log file and it's tracker.
`private long`	`initOldLogs(org.apache.hadoop.fs.FileStatus[] logFiles)` Deprecated. Make sure that the file set are gotten by calling `getLogFiles()`, where we will sort the file set by log id.
`private void`	`initTrackerFromOldLogs()` Deprecated. If last log's tracker is not null, use it as `storeTracker`.
`void`	`insert(Procedure<?>[] procs)` Deprecated. Serialize a set of new procedures.
`void`	`insert(Procedure<?> proc, Procedure<?>[] subprocs)` Deprecated. When a procedure is submitted to the executor insert(proc, null) will be called.
`private boolean`	`isSyncAborted()` Deprecated.
`void`	`load(ProcedureStore.ProcedureLoader loader)` Deprecated. Load the Procedures in the store.
`static void`	`main(String[] args)` Deprecated. Parses a directory of WALs building up ProcedureState.
`private void`	`periodicRoll()` Deprecated.
`(package private) void`	`periodicRollForTesting()` Deprecated.
`private long`	`pushData(WALProcedureStore.PushType type, ByteSlot slot, long procId, long[] subProcIds)` Deprecated.
`void`	`recoverLease()` Deprecated. Acquire the lease for the procedure store.
`private void`	`releaseSlot(ByteSlot slot)` Deprecated.
`private void`	`removeAllLogs(long lastLogId, String why)` Deprecated. Remove all logs with logId <= `lastLogId`.
`private void`	`removeInactiveLogs()` Deprecated.
`(package private) void`	`removeInactiveLogsForTesting()` Deprecated.
`private boolean`	`removeLogFile(ProcedureWALFile log, org.apache.hadoop.fs.Path walArchiveDir)` Deprecated.
`private boolean`	`rollWriter()` Deprecated.
`(package private) boolean`	`rollWriter(long logId)` Deprecated.
`boolean`	`rollWriterForTesting()` Deprecated.
`private boolean`	`rollWriterWithRetries()` Deprecated.
`private void`	`sendStopSignal()` Deprecated.
`int`	`setRunningProcedureCount(int count)` Deprecated. Set the number of procedure running.
`void`	`start(int numSlots)` Deprecated. Start/Open the procedure store
`void`	`stop(boolean abort)` Deprecated. Stop/Close the procedure store
`private void`	`syncLoop()` Deprecated.
`private long`	`syncSlots()` Deprecated.
`protected long`	`syncSlots(org.apache.hadoop.fs.FSDataOutputStream stream, ByteSlot[] slots, int offset, int count)` Deprecated.
`protected void`	`syncStream(org.apache.hadoop.fs.FSDataOutputStream stream)` Deprecated.
`private void`	`tryCleanupLogsOnLoad()` Deprecated.
`private boolean`	`tryRollWriter()` Deprecated.
`void`	`update(Procedure<?> proc)` Deprecated. The specified procedure was executed, and the new state should be written to the store.
`private void`	`updateStoreTracker(WALProcedureStore.PushType type, long procId, long[] subProcIds)` Deprecated.

Methods inherited from class org.apache.hadoop.hbase.procedure2.store.ProcedureStoreBase
isRunning, registerListener, sendAbortProcessSignal, sendForceUpdateSignal, sendPostSyncSignal, setRunning, unregisterListener

Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface org.apache.hadoop.hbase.procedure2.store.ProcedureStore
cleanup

- Field Detail
  - LOG
```
private static final org.slf4j.Logger LOG
```
    Deprecated.
  - LOG_PREFIX
```
public static final String LOG_PREFIX
```
    Deprecated.
    
    See Also:
    
    Constant Field Values
  - MASTER_PROCEDURE_LOGDIR
```
public static final String MASTER_PROCEDURE_LOGDIR
```
    Deprecated.
    
    Used to construct the name of the log directory for master procedures
    
    See Also:
    
    Constant Field Values
  - WAL_COUNT_WARN_THRESHOLD_CONF_KEY
```
public static final String WAL_COUNT_WARN_THRESHOLD_CONF_KEY
```
    Deprecated.
    
    See Also:
    
    Constant Field Values
  - DEFAULT_WAL_COUNT_WARN_THRESHOLD
```
private static final int DEFAULT_WAL_COUNT_WARN_THRESHOLD
```
    Deprecated.
    
    See Also:
    
    Constant Field Values
  - EXEC_WAL_CLEANUP_ON_LOAD_CONF_KEY
```
public static final String EXEC_WAL_CLEANUP_ON_LOAD_CONF_KEY
```
    Deprecated.
    
    See Also:
    
    Constant Field Values
  - DEFAULT_EXEC_WAL_CLEANUP_ON_LOAD_CONF_KEY
```
private static final boolean DEFAULT_EXEC_WAL_CLEANUP_ON_LOAD_CONF_KEY
```
    Deprecated.
    
    See Also:
    
    Constant Field Values
  - MAX_RETRIES_BEFORE_ROLL_CONF_KEY
```
public static final String MAX_RETRIES_BEFORE_ROLL_CONF_KEY
```
    Deprecated.
    
    See Also:
    
    Constant Field Values
  - DEFAULT_MAX_RETRIES_BEFORE_ROLL
```
private static final int DEFAULT_MAX_RETRIES_BEFORE_ROLL
```
    Deprecated.
    
    See Also:
    
    Constant Field Values
  - WAIT_BEFORE_ROLL_CONF_KEY
```
public static final String WAIT_BEFORE_ROLL_CONF_KEY
```
    Deprecated.
    
    See Also:
    
    Constant Field Values
  - DEFAULT_WAIT_BEFORE_ROLL
```
private static final int DEFAULT_WAIT_BEFORE_ROLL
```
    Deprecated.
    
    See Also:
    
    Constant Field Values
  - ROLL_RETRIES_CONF_KEY
```
public static final String ROLL_RETRIES_CONF_KEY
```
    Deprecated.
    
    See Also:
    
    Constant Field Values
  - DEFAULT_ROLL_RETRIES
```
private static final int DEFAULT_ROLL_RETRIES
```
    Deprecated.
    
    See Also:
    
    Constant Field Values
  - MAX_SYNC_FAILURE_ROLL_CONF_KEY
```
public static final String MAX_SYNC_FAILURE_ROLL_CONF_KEY
```
    Deprecated.
    
    See Also:
    
    Constant Field Values
  - DEFAULT_MAX_SYNC_FAILURE_ROLL
```
private static final int DEFAULT_MAX_SYNC_FAILURE_ROLL
```
    Deprecated.
    
    See Also:
    
    Constant Field Values
  - PERIODIC_ROLL_CONF_KEY
```
public static final String PERIODIC_ROLL_CONF_KEY
```
    Deprecated.
    
    See Also:
    
    Constant Field Values
  - DEFAULT_PERIODIC_ROLL
```
private static final int DEFAULT_PERIODIC_ROLL
```
    Deprecated.
    
    See Also:
    
    Constant Field Values
  - SYNC_WAIT_MSEC_CONF_KEY
```
public static final String SYNC_WAIT_MSEC_CONF_KEY
```
    Deprecated.
    
    See Also:
    
    Constant Field Values
  - DEFAULT_SYNC_WAIT_MSEC
```
private static final int DEFAULT_SYNC_WAIT_MSEC
```
    Deprecated.
    
    See Also:
    
    Constant Field Values
  - USE_HSYNC_CONF_KEY
```
public static final String USE_HSYNC_CONF_KEY
```
    Deprecated.
    
    See Also:
    
    Constant Field Values
  - DEFAULT_USE_HSYNC
```
private static final boolean DEFAULT_USE_HSYNC
```
    Deprecated.
    
    See Also:
    
    Constant Field Values
  - ROLL_THRESHOLD_CONF_KEY
```
public static final String ROLL_THRESHOLD_CONF_KEY
```
    Deprecated.
    
    See Also:
    
    Constant Field Values
  - DEFAULT_ROLL_THRESHOLD
```
private static final long DEFAULT_ROLL_THRESHOLD
```
    Deprecated.
    
    See Also:
    
    Constant Field Values
  - STORE_WAL_SYNC_STATS_COUNT
```
public static final String STORE_WAL_SYNC_STATS_COUNT
```
    Deprecated.
    
    See Also:
    
    Constant Field Values
  - DEFAULT_SYNC_STATS_COUNT
```
private static final int DEFAULT_SYNC_STATS_COUNT
```
    Deprecated.
    
    See Also:
    
    Constant Field Values
  - logs
```
private final LinkedList<ProcedureWALFile> logs
```
    Deprecated.
  - holdingCleanupTracker
```
private final ProcedureStoreTracker holdingCleanupTracker
```
    Deprecated.
  - storeTracker
```
private final ProcedureStoreTracker storeTracker
```
    Deprecated.
  - lock
```
private final ReentrantLock lock
```
    Deprecated.
  - waitCond
```
private final Condition waitCond
```
    Deprecated.
  - slotCond
```
private final Condition slotCond
```
    Deprecated.
  - syncCond
```
private final Condition syncCond
```
    Deprecated.
  - leaseRecovery
```
private final LeaseRecovery leaseRecovery
```
    Deprecated.
  - conf
```
private final org.apache.hadoop.conf.Configuration conf
```
    Deprecated.
  - fs
```
private final org.apache.hadoop.fs.FileSystem fs
```
    Deprecated.
  - walDir
```
private final org.apache.hadoop.fs.Path walDir
```
    Deprecated.
  - walArchiveDir
```
private final org.apache.hadoop.fs.Path walArchiveDir
```
    Deprecated.
  - enforceStreamCapability
```
private final boolean enforceStreamCapability
```
    Deprecated.
  - syncException
```
private final AtomicReference<Throwable> syncException
```
    Deprecated.
  - loading
```
private final AtomicBoolean loading
```
    Deprecated.
  - inSync
```
private final AtomicBoolean inSync
```
    Deprecated.
  - totalSynced
```
private final AtomicLong totalSynced
```
    Deprecated.
  - lastRollTs
```
private final AtomicLong lastRollTs
```
    Deprecated.
  - syncId
```
private final AtomicLong syncId
```
    Deprecated.
  - slotsCache
```
private LinkedTransferQueue<ByteSlot> slotsCache
```
    Deprecated.
  - corruptedLogs
```
private Set<ProcedureWALFile> corruptedLogs
```
    Deprecated.
  - stream
```
private org.apache.hadoop.fs.FSDataOutputStream stream
```
    Deprecated.
  - runningProcCount
```
private int runningProcCount
```
    Deprecated.
  - flushLogId
```
private long flushLogId
```
    Deprecated.
  - syncMaxSlot
```
private int syncMaxSlot
```
    Deprecated.
  - slotIndex
```
private int slotIndex
```
    Deprecated.
  - syncThread
```
private Thread syncThread
```
    Deprecated.
  - slots
```
private ByteSlot[] slots
```
    Deprecated.
  - walCountWarnThreshold
```
private int walCountWarnThreshold
```
    Deprecated.
  - maxRetriesBeforeRoll
```
private int maxRetriesBeforeRoll
```
    Deprecated.
  - maxSyncFailureRoll
```
private int maxSyncFailureRoll
```
    Deprecated.
  - waitBeforeRoll
```
private int waitBeforeRoll
```
    Deprecated.
  - rollRetries
```
private int rollRetries
```
    Deprecated.
  - periodicRollMsec
```
private int periodicRollMsec
```
    Deprecated.
  - rollThreshold
```
private long rollThreshold
```
    Deprecated.
  - useHsync
```
private boolean useHsync
```
    Deprecated.
  - syncWaitMsec
```
private int syncWaitMsec
```
    Deprecated.
  - syncMetricsQueue
```
private org.apache.hbase.thirdparty.org.apache.commons.collections4.queue.CircularFifoQueue<WALProcedureStore.SyncMetrics> syncMetricsQueue
```
    Deprecated.
  - WALS_PATH_FILTER
```
private static final org.apache.hadoop.fs.PathFilter WALS_PATH_FILTER
```
    Deprecated.
  - FILE_STATUS_ID_COMPARATOR
```
private static final Comparator<org.apache.hadoop.fs.FileStatus> FILE_STATUS_ID_COMPARATOR
```
    Deprecated.
- Constructor Detail
  - WALProcedureStore
```
public WALProcedureStore(org.apache.hadoop.conf.Configuration conf,
                         LeaseRecovery leaseRecovery)
                  throws IOException
```
    Deprecated.
    
    Throws:
    
    IOException
  - WALProcedureStore
```
public WALProcedureStore(org.apache.hadoop.conf.Configuration conf,
                         org.apache.hadoop.fs.Path walDir,
                         org.apache.hadoop.fs.Path walArchiveDir,
                         LeaseRecovery leaseRecovery)
                  throws IOException
```
    Deprecated.
    
    Throws:
    
    IOException
- Method Detail
  - start
```
public void start(int numSlots)
           throws IOException
```
    Deprecated.
    
    Description copied from interface: ProcedureStore
    
    Start/Open the procedure store
    
    Parameters:
    
    numSlots - number of threads to be used by the procedure store
    
    Throws:
    
    IOException
  - stop
```
public void stop(boolean abort)
```
    Deprecated.
    
    Description copied from interface: ProcedureStore
    
    Stop/Close the procedure store
    
    Parameters:
    
    abort - true if the stop is an abort
  - sendStopSignal
```
private void sendStopSignal()
```
    Deprecated.
  - getNumThreads
```
public int getNumThreads()
```
    Deprecated.
    
    Description copied from interface: ProcedureStore
    
    Returns the number of threads/slots passed to start()
  - setRunningProcedureCount
```
public int setRunningProcedureCount(int count)
```
    Deprecated.
    
    Description copied from interface: ProcedureStore
    
    Set the number of procedure running. This can be used, for example, by the store to know how long to wait before a sync.
    
    Returns:
    
    how many procedures are running (may not be same as count).
  - getStoreTracker
```
public ProcedureStoreTracker getStoreTracker()
```
    Deprecated.
  - getActiveLogs
```
public ArrayList<ProcedureWALFile> getActiveLogs()
```
    Deprecated.
  - getCorruptedLogs
```
public Set<ProcedureWALFile> getCorruptedLogs()
```
    Deprecated.
  - recoverLease
```
public void recoverLease()
                  throws IOException
```
    Deprecated.
    
    Description copied from interface: ProcedureStore
    
    Acquire the lease for the procedure store.
    
    Throws:
    
    IOException
  - load
```
public void load(ProcedureStore.ProcedureLoader loader)
          throws IOException
```
    Deprecated.
    
    Description copied from interface: ProcedureStore
    
    Load the Procedures in the store.
    
    Parameters:
    
    loader - the ProcedureLoader that will handle the store-load events
    
    Throws:
    
    IOException
  - tryCleanupLogsOnLoad
```
private void tryCleanupLogsOnLoad()
```
    Deprecated.
  - insert
```
public void insert(Procedure<?> proc,
                   Procedure<?>[] subprocs)
```
    Deprecated.
    
    Description copied from interface: ProcedureStore
    
    When a procedure is submitted to the executor insert(proc, null) will be called. 'proc' has a 'RUNNABLE' state and the initial information required to start up. When a procedure is executed and it returns children insert(proc, subprocs) will be called. 'proc' has a 'WAITING' state and an update state. 'subprocs' are the children in 'RUNNABLE' state with the initial information.
    
    Parameters:
    
    proc - the procedure to serialize and write to the store.
    
    subprocs - the newly created child of the proc.
  - insert
```
public void insert(Procedure<?>[] procs)
```
    Deprecated.
    
    Description copied from interface: ProcedureStore
    
    Serialize a set of new procedures. These procedures are freshly submitted to the executor and each procedure has a 'RUNNABLE' state and the initial information required to start up.
    
    Parameters:
    
    procs - the procedures to serialize and write to the store.
  - update
```
public void update(Procedure<?> proc)
```
    Deprecated.
    
    Description copied from interface: ProcedureStore
    
    The specified procedure was executed, and the new state should be written to the store.
    
    Parameters:
    
    proc - the procedure to serialize and write to the store.
  - delete
```
public void delete(long procId)
```
    Deprecated.
    
    Description copied from interface: ProcedureStore
    
    The specified procId was removed from the executor, due to completion, abort or failure. The store implementor should remove all the information about the specified procId.
    
    Parameters:
    
    procId - the ID of the procedure to remove.
  - delete
```
public void delete(Procedure<?> proc,
                   long[] subProcIds)
```
    Deprecated.
    
    Description copied from interface: ProcedureStore
    
    The parent procedure completed. Update the state and mark all the child deleted.
    
    Parameters:
    
    proc - the parent procedure to serialize and write to the store.
    
    subProcIds - the IDs of the sub-procedure to remove.
  - delete
```
public void delete(long[] procIds,
                   int offset,
                   int count)
```
    Deprecated.
    
    Description copied from interface: ProcedureStore
    
    The specified procIds were removed from the executor, due to completion, abort or failure. The store implementor should remove all the information about the specified procIds.
    
    Parameters:
    
    procIds - the IDs of the procedures to remove.
    
    offset - the array offset from where to start to delete
    
    count - the number of IDs to delete
  - delete
```
private void delete(long[] procIds)
```
    Deprecated.
  - acquireSlot
```
private ByteSlot acquireSlot()
```
    Deprecated.
  - releaseSlot
```
private void releaseSlot(ByteSlot slot)
```
    Deprecated.
  - pushData
```
private long pushData(WALProcedureStore.PushType type,
                      ByteSlot slot,
                      long procId,
                      long[] subProcIds)
```
    Deprecated.
  - updateStoreTracker
```
private void updateStoreTracker(WALProcedureStore.PushType type,
                                long procId,
                                long[] subProcIds)
```
    Deprecated.
  - isSyncAborted
```
private boolean isSyncAborted()
```
    Deprecated.
  - syncLoop
```
private void syncLoop()
               throws Throwable
```
    Deprecated.
    
    Throws:
    
    Throwable
  - getSyncMetrics
```
public ArrayList<WALProcedureStore.SyncMetrics> getSyncMetrics()
```
    Deprecated.
  - syncSlots
```
private long syncSlots()
                throws Throwable
```
    Deprecated.
    
    Throws:
    
    Throwable
  - syncSlots
```
protected long syncSlots(org.apache.hadoop.fs.FSDataOutputStream stream,
                         ByteSlot[] slots,
                         int offset,
                         int count)
                  throws IOException
```
    Deprecated.
    
    Throws:
    
    IOException
  - syncStream
```
protected void syncStream(org.apache.hadoop.fs.FSDataOutputStream stream)
                   throws IOException
```
    Deprecated.
    
    Throws:
    
    IOException
  - rollWriterWithRetries
```
private boolean rollWriterWithRetries()
```
    Deprecated.
  - tryRollWriter
```
private boolean tryRollWriter()
```
    Deprecated.
  - getMillisToNextPeriodicRoll
```
public long getMillisToNextPeriodicRoll()
```
    Deprecated.
  - getMillisFromLastRoll
```
public long getMillisFromLastRoll()
```
    Deprecated.
  - periodicRollForTesting
```
void periodicRollForTesting()
                     throws IOException
```
    Deprecated.
    
    Throws:
    
    IOException
  - rollWriterForTesting
```
public boolean rollWriterForTesting()
                             throws IOException
```
    Deprecated.
    
    Throws:
    
    IOException
  - removeInactiveLogsForTesting
```
void removeInactiveLogsForTesting()
                           throws Exception
```
    Deprecated.
    
    Throws:
    
    Exception
  - periodicRoll
```
private void periodicRoll()
                   throws IOException
```
    Deprecated.
    
    Throws:
    
    IOException
  - rollWriter
```
private boolean rollWriter()
                    throws IOException
```
    Deprecated.
    
    Throws:
    
    IOException
  - rollWriter
```
boolean rollWriter(long logId)
            throws IOException
```
    Deprecated.
    
    Throws:
    
    IOException
  - closeCurrentLogStream
```
private void closeCurrentLogStream(boolean abort)
```
    Deprecated.
  - removeInactiveLogs
```
private void removeInactiveLogs()
                         throws IOException
```
    Deprecated.
    
    Throws:
    
    IOException
  - buildHoldingCleanupTracker
```
private void buildHoldingCleanupTracker()
```
    Deprecated.
  - removeAllLogs
```
private void removeAllLogs(long lastLogId,
                           String why)
```
    Deprecated.
    
    Remove all logs with logId <= lastLogId.
  - removeLogFile
```
private boolean removeLogFile(ProcedureWALFile log,
                              org.apache.hadoop.fs.Path walArchiveDir)
```
    Deprecated.
  - getWALDir
```
public org.apache.hadoop.fs.Path getWALDir()
```
    Deprecated.
  - getWalArchiveDir
```
org.apache.hadoop.fs.Path getWalArchiveDir()
```
    Deprecated.
  - getFileSystem
```
public org.apache.hadoop.fs.FileSystem getFileSystem()
```
    Deprecated.
  - getLogFilePath
```
protected org.apache.hadoop.fs.Path getLogFilePath(long logId)
                                            throws IOException
```
    Deprecated.
    
    Throws:
    
    IOException
  - getLogIdFromName
```
private static long getLogIdFromName(String name)
```
    Deprecated.
  - getLogFiles
```
private org.apache.hadoop.fs.FileStatus[] getLogFiles()
                                               throws IOException
```
    Deprecated.
    
    Throws:
    
    IOException
  - getMaxLogId
```
private static long getMaxLogId(org.apache.hadoop.fs.FileStatus[] logFiles)
```
    Deprecated.
    
    Make sure that the file set are gotten by calling getLogFiles(), where we will sort the file set by log id.
    
    Returns:
    
    Max-LogID of the specified log file set
  - initOldLogs
```
private long initOldLogs(org.apache.hadoop.fs.FileStatus[] logFiles)
                  throws IOException
```
    Deprecated.
    
    Make sure that the file set are gotten by calling getLogFiles(), where we will sort the file set by log id.
    
    Returns:
    
    Max-LogID of the specified log file set
    
    Throws:
    
    IOException
  - initTrackerFromOldLogs
```
private void initTrackerFromOldLogs()
```
    Deprecated.
    
    If last log's tracker is not null, use it as storeTracker. Otherwise, set storeTracker as partial, and let ProcedureWALFormatReader rebuild it using entries in the log.
  - initOldLog
```
private ProcedureWALFile initOldLog(org.apache.hadoop.fs.FileStatus logFile,
                                    org.apache.hadoop.fs.Path walArchiveDir)
                             throws IOException
```
    Deprecated.
    
    Loads given log file and it's tracker.
    
    Throws:
    
    IOException
  - main
```
public static void main(String[] args)
                 throws IOException
```
    Deprecated.
    
    Parses a directory of WALs building up ProcedureState. For testing parse and profiling.
    
    Parameters:
    
    args - Include pointer to directory of WAL files for a store instance to parse & load.
    
    Throws:
    
    IOException

Class WALProcedureStore

Nested Class Summary

Nested classes/interfaces inherited from interface org.apache.hadoop.hbase.procedure2.store.ProcedureStore

Field Summary

Constructor Summary

Method Summary

Methods inherited from class org.apache.hadoop.hbase.procedure2.store.ProcedureStoreBase

Methods inherited from class java.lang.Object

Methods inherited from interface org.apache.hadoop.hbase.procedure2.store.ProcedureStore

Field Detail

LOG

LOG_PREFIX

MASTER_PROCEDURE_LOGDIR

WAL_COUNT_WARN_THRESHOLD_CONF_KEY

DEFAULT_WAL_COUNT_WARN_THRESHOLD

EXEC_WAL_CLEANUP_ON_LOAD_CONF_KEY

DEFAULT_EXEC_WAL_CLEANUP_ON_LOAD_CONF_KEY

MAX_RETRIES_BEFORE_ROLL_CONF_KEY

DEFAULT_MAX_RETRIES_BEFORE_ROLL

WAIT_BEFORE_ROLL_CONF_KEY

DEFAULT_WAIT_BEFORE_ROLL

ROLL_RETRIES_CONF_KEY

DEFAULT_ROLL_RETRIES

MAX_SYNC_FAILURE_ROLL_CONF_KEY

DEFAULT_MAX_SYNC_FAILURE_ROLL

PERIODIC_ROLL_CONF_KEY

DEFAULT_PERIODIC_ROLL

SYNC_WAIT_MSEC_CONF_KEY

DEFAULT_SYNC_WAIT_MSEC

USE_HSYNC_CONF_KEY

DEFAULT_USE_HSYNC

ROLL_THRESHOLD_CONF_KEY

DEFAULT_ROLL_THRESHOLD

STORE_WAL_SYNC_STATS_COUNT

DEFAULT_SYNC_STATS_COUNT

logs

holdingCleanupTracker

storeTracker

lock

waitCond

slotCond

syncCond

leaseRecovery

conf

fs

walDir

walArchiveDir

enforceStreamCapability

syncException

loading

inSync

totalSynced

lastRollTs

syncId

slotsCache

corruptedLogs

stream

runningProcCount

flushLogId

syncMaxSlot

slotIndex

syncThread

slots

walCountWarnThreshold

maxRetriesBeforeRoll

maxSyncFailureRoll

waitBeforeRoll

rollRetries

periodicRollMsec

rollThreshold

useHsync

syncWaitMsec

syncMetricsQueue

WALS_PATH_FILTER

FILE_STATUS_ID_COMPARATOR

Constructor Detail

WALProcedureStore

WALProcedureStore

Method Detail

start