org.apache.hadoop.hbase.wal

Interface WAL

All Superinterfaces:

AutoCloseable, Closeable, WALFileLengthProvider

All Known Implementing Classes:

AbstractFSWAL, AsyncFSWAL, DisabledWALProvider.DisabledWAL, FSHLog

@InterfaceAudience.Private
 @InterfaceStability.Evolving
public interface WAL
extends Closeable, WALFileLengthProvider

A Write Ahead Log (WAL) provides service for reading, writing waledits. This interface provides APIs for WAL users (such as RegionServer) to use the WAL (do append, sync, etc). Note that some internals, such as log rolling and performance evaluation tools, will use WAL.equals to determine if they have already seen a given WAL.

Nested Class Summary

Nested Classes

Modifier and Type	Interface and Description
static class	WAL.Entry Utility class that lets us keep track of the edit with it's key.
static interface	WAL.Reader When outside clients need to consume persisted WALs, they rely on a provided Reader.

Method Summary

Modifier and Type	Method and Description
void	abortCacheFlush(byte[] encodedRegionName) Abort a cache flush.
long	appendData(RegionInfo info, WALKeyImpl key, WALEdit edits) Append a set of data edits to the WAL.
long	appendMarker(RegionInfo info, WALKeyImpl key, WALEdit edits) Append an operational 'meta' event marker edit to the WAL.
void	close() Caller no longer needs any edits from this WAL.
void	completeCacheFlush(byte[] encodedRegionName, long maxFlushedSeqId) Complete the cache flush.
WALCoprocessorHost	getCoprocessorHost() Returns Coprocessor host.
long	getEarliestMemStoreSeqNum(byte[] encodedRegionName) Deprecated. Since version 1.2.0. Removing because not used and exposes subtle internal workings. Use getEarliestMemStoreSeqNum(byte[], byte[])
long	getEarliestMemStoreSeqNum(byte[] encodedRegionName, byte[] familyName) Gets the earliest unflushed sequence id in the memstore for the store.
static long	getTimestamp(String name) Split a WAL filename to get a start time.
void	registerWALActionsListener(WALActionsListener listener) Registers WALActionsListener
Map<byte[],List<byte[]>>	rollWriter() Roll the log writer.
Map<byte[],List<byte[]>>	rollWriter(boolean force) Roll the log writer.
void	shutdown() Stop accepting new writes.
Long	startCacheFlush(byte[] encodedRegionName, Map<byte[],Long> familyToSeq)
Long	startCacheFlush(byte[] encodedRegionName, Set<byte[]> families) WAL keeps track of the sequence numbers that are as yet not flushed im memstores in order to be able to do accounting to figure which WALs can be let go.
void	sync() Sync what we have in the WAL.
default void	sync(boolean forceSync)
void	sync(long txid) Sync the WAL if the txId was not already sync'd.
default void	sync(long txid, boolean forceSync)
String	toString() Human readable identifying information about the state of this WAL.
boolean	unregisterWALActionsListener(WALActionsListener listener) Unregisters WALActionsListener
void	updateStore(byte[] encodedRegionName, byte[] familyName, Long sequenceid, boolean onlyIfGreater) updates the seuence number of a specific store.

Methods inherited from interface org.apache.hadoop.hbase.replication.regionserver.WALFileLengthProvider

getLogFileSizeIfBeingWritten

Method Detail

registerWALActionsListener

void registerWALActionsListener(WALActionsListener listener)

Registers WALActionsListener

unregisterWALActionsListener

boolean unregisterWALActionsListener(WALActionsListener listener)

Unregisters WALActionsListener

rollWriter

Map<byte[],List<byte[]>> rollWriter()
                             throws FailedLogCloseException,
                                    IOException

Roll the log writer. That is, start writing log messages to a new file.

The implementation is synchronized in order to make sure there's one rollWriter running at any given time.

Returns:

If lots of logs, flush the stores of returned regions so next time through we can clean logs. Returns null if nothing to flush. Names are actual region names as returned by RegionInfo.getEncodedName()

Throws:

FailedLogCloseException

IOException

rollWriter

Map<byte[],List<byte[]>> rollWriter(boolean force)
                             throws IOException

Roll the log writer. That is, start writing log messages to a new file.

The implementation is synchronized in order to make sure there's one rollWriter running at any given time. n * If true, force creation of a new writer even if no entries have been written to the current writer

Returns:

If lots of logs, flush the stores of returned regions so next time through we can clean logs. Returns null if nothing to flush. Names are actual region names as returned by RegionInfo.getEncodedName()

Throws:

IOException

shutdown

void shutdown()
       throws IOException

Stop accepting new writes. If we have unsynced writes still in buffer, sync them. Extant edits are left in place in backing storage to be replayed later.

Throws:

IOException

close

void close()
    throws IOException

Caller no longer needs any edits from this WAL. Implementers are free to reclaim underlying resources after this call; i.e. filesystem based WALs can archive or delete files.

Specified by:

close in interface AutoCloseable

Specified by:

close in interface Closeable

Throws:

IOException

appendData

long appendData(RegionInfo info,
                WALKeyImpl key,
                WALEdit edits)
         throws IOException

Append a set of data edits to the WAL. 'Data' here means that the content in the edits will also have transitioned through the memstore.

The WAL is not flushed/sync'd after this transaction completes BUT on return this edit must have its region edit/sequence id assigned else it messes up our unification of mvcc and sequenceid. On return key will have the region edit/sequence id filled in.

Parameters:

info - the regioninfo associated with append

key - Modified by this call; we add to it this edits region edit/sequence id.

edits - Edits to append. MAY CONTAIN NO EDITS for case where we want to get an edit sequence id that is after all currently appended edits.

Returns:

Returns a 'transaction id' and key will have the region edit/sequence id in it.

Throws:

IOException

See Also:

appendMarker(RegionInfo, WALKeyImpl, WALEdit)

appendMarker

long appendMarker(RegionInfo info,
                  WALKeyImpl key,
                  WALEdit edits)
           throws IOException

Append an operational 'meta' event marker edit to the WAL. A marker meta edit could be a FlushDescriptor, a compaction marker, or a region event marker; e.g. region open or region close. The difference between a 'marker' append and a 'data' append as in appendData(RegionInfo, WALKeyImpl, WALEdit)is that a marker will not have transitioned through the memstore.

The WAL is not flushed/sync'd after this transaction completes BUT on return this edit must have its region edit/sequence id assigned else it messes up our unification of mvcc and sequenceid. On return key will have the region edit/sequence id filled in.

Parameters:

info - the regioninfo associated with append

key - Modified by this call; we add to it this edits region edit/sequence id.

edits - Edits to append. MAY CONTAIN NO EDITS for case where we want to get an edit sequence id that is after all currently appended edits.

Returns:

Returns a 'transaction id' and key will have the region edit/sequence id in it.

Throws:

IOException

See Also:

appendData(RegionInfo, WALKeyImpl, WALEdit)

updateStore

void updateStore(byte[] encodedRegionName,
                 byte[] familyName,
                 Long sequenceid,
                 boolean onlyIfGreater)

updates the seuence number of a specific store. depending on the flag: replaces current seq number if the given seq id is bigger, or even if it is lower than existing one

sync

void sync()
   throws IOException

Sync what we have in the WAL.

Throws:

IOException

sync

void sync(long txid)
   throws IOException

Sync the WAL if the txId was not already sync'd.

Parameters:

txid - Transaction id to sync to.

Throws:

IOException

sync

default void sync(boolean forceSync)
           throws IOException

Parameters:

forceSync - Flag to force sync rather than flushing to the buffer. Example - Hadoop hflush vs hsync.

Throws:

IOException

sync

default void sync(long txid,
                  boolean forceSync)
           throws IOException

Parameters:

txid - Transaction id to sync to.

forceSync - Flag to force sync rather than flushing to the buffer. Example - Hadoop hflush vs hsync.

Throws:

IOException

startCacheFlush

Long startCacheFlush(byte[] encodedRegionName,
                     Set<byte[]> families)

WAL keeps track of the sequence numbers that are as yet not flushed im memstores in order to be able to do accounting to figure which WALs can be let go. This method tells WAL that some region is about to flush. The flush can be the whole region or for a column family of the region only.

Currently, it is expected that the update lock is held for the region; i.e. no concurrent appends while we set up cache flush.

Parameters:

families - Families to flush. May be a subset of all families in the region.

Returns:

Returns HConstants.NO_SEQNUM if we are flushing the whole region OR if we are flushing a subset of all families but there are no edits in those families not being flushed; in other words, this is effectively same as a flush of all of the region though we were passed a subset of regions. Otherwise, it returns the sequence id of the oldest/lowest outstanding edit.

See Also:

completeCacheFlush(byte[], long), abortCacheFlush(byte[])

startCacheFlush

Long startCacheFlush(byte[] encodedRegionName,
                     Map<byte[],Long> familyToSeq)

completeCacheFlush

void completeCacheFlush(byte[] encodedRegionName,
                        long maxFlushedSeqId)

Complete the cache flush.

Parameters:

encodedRegionName - Encoded region name.

maxFlushedSeqId - The maxFlushedSeqId for this flush. There is no edit in memory that is less that this sequence id.

See Also:

startCacheFlush(byte[], Set), abortCacheFlush(byte[])

abortCacheFlush

void abortCacheFlush(byte[] encodedRegionName)

Abort a cache flush. Call if the flush fails. Note that the only recovery for an aborted flush currently is a restart of the regionserver so the snapshot content dropped by the failure gets restored to the memstore.

Parameters:

encodedRegionName - Encoded region name.

getCoprocessorHost

WALCoprocessorHost getCoprocessorHost()

Returns Coprocessor host.

getEarliestMemStoreSeqNum

@Deprecated
long getEarliestMemStoreSeqNum(byte[] encodedRegionName)

Deprecated. Since version 1.2.0. Removing because not used and exposes subtle internal workings. Use getEarliestMemStoreSeqNum(byte[], byte[])

Gets the earliest unflushed sequence id in the memstore for the region.

Parameters:

encodedRegionName - The region to get the number for.

Returns:

The earliest/lowest/oldest sequence id if present, HConstants.NO_SEQNUM if absent.

getEarliestMemStoreSeqNum

long getEarliestMemStoreSeqNum(byte[] encodedRegionName,
                               byte[] familyName)

Gets the earliest unflushed sequence id in the memstore for the store.

Parameters:

encodedRegionName - The region to get the number for.

familyName - The family to get the number for.

Returns:

The earliest/lowest/oldest sequence id if present, HConstants.NO_SEQNUM if absent.

toString

String toString()

Human readable identifying information about the state of this WAL. Implementors are encouraged to include information appropriate for debugging. Consumers are advised not to rely on the details of the returned String; it does not have a defined structure.

Overrides:

toString in class Object

getTimestamp

static long getTimestamp(String name)

Split a WAL filename to get a start time. WALs usually have the time we start writing to them as part of their name, usually the suffix. Sometimes there will be an extra suffix as when it is a WAL for the meta table. For example, WALs might look like this 10.20.20.171%3A60020.1277499063250 where 1277499063250 is the timestamp. Could also be a meta WAL which adds a '.meta' suffix or a synchronous replication WAL which adds a '.syncrep' suffix. Check for these. File also may have no timestamp on it. For example the recovered.edits files are WALs but are named in ascending order. Here is an example: 0000000000000016310. Allow for this.

Parameters:

name - Name of the WAL file.

Returns:

Timestamp or -1.