Package org.apache.hadoop.hbase.wal

Class AbstractWALProvider

java.lang.Object

org.apache.hadoop.hbase.wal.AbstractWALProvider

All Implemented Interfaces:

PeerActionListener, WALProvider

Direct Known Subclasses:

AbstractFSWALProvider, RegionGroupingProvider

@Private
public abstract class AbstractWALProvider
extends Object
implements WALProvider, PeerActionListener

Base class for a WAL Provider.

We will put some common logic here, especially for sync replication implementation, as it must do some hacks before the normal wal creation operation.

All WALProvider implementations should extends this class instead of implement WALProvider directly, except DisabledWALProvider.

Nested Class Summary

Nested classes/interfaces inherited from interface org.apache.hadoop.hbase.wal.WALProvider

WALProvider.AsyncWriter, WALProvider.Writer, WALProvider.WriterBase

Field Summary

Fields

Modifier and Type	Field	Description
protected Abortable	abortable
protected org.apache.hadoop.conf.Configuration	conf
private final KeyLocker<String>	createLock
protected WALFactory	factory
protected AtomicBoolean	initialized
protected List<WALActionsListener>	listeners
private static final org.slf4j.Logger	LOG
private static final Pattern	LOG_PREFIX_PATTERN
protected String	logPrefix
private final Condition	noRemoteWALUnderCreationCond
private int	numRemoteWALUnderCreation
private final Lock	numRemoteWALUnderCreationLock
private final ConcurrentMap<String,Optional<WAL>>	peerId2WAL
private SyncReplicationPeerInfoProvider	peerInfoProvider
protected String	providerId
static final String	WAL_FILE_NAME_DELIMITER

Fields inherited from interface org.apache.hadoop.hbase.replication.regionserver.PeerActionListener

DUMMY

Constructor Summary

Constructors

Constructor	Description
AbstractWALProvider()

Method Summary

Modifier and Type	Method	Description
void	addWALActionsListener(WALActionsListener listener)	Add a WALActionsListener.
private void	cleanup(IOExceptionConsumer<WAL> cleanupWAL, IOExceptionRunnable finalCleanup)
final void	close()	shutdown utstanding WALs and clean up any persisted state.
protected abstract void	close0()
protected abstract WAL	createRemoteWAL(RegionInfo region, org.apache.hadoop.fs.FileSystem remoteFs, org.apache.hadoop.fs.Path remoteWALDir, String prefix, String suffix)
protected abstract void	doInit(WALFactory factory, org.apache.hadoop.conf.Configuration conf, String providerId)
final long	getLogFileSize()	Get size of the log files this provider is managing
protected abstract long	getLogFileSize0()
final long	getNumLogFiles()	Get number of the log files this provider is managing
protected abstract long	getNumLogFiles0()
PeerActionListener	getPeerActionListener()
private WAL	getRemoteWAL(RegionInfo region, String peerId, String remoteWALDir)
private String	getRemoteWALPrefix(String peerId)
static Optional<String>	getSyncReplicationPeerIdFromWALName(String name)	Returns the peer id if the wal file name is in the special group for a sync replication peer.
final WAL	getWAL(RegionInfo region)
protected abstract WAL	getWAL0(RegionInfo region)
final List<WAL>	getWALs()	Returns the List of WALs that are used by this server
protected abstract List<WAL>	getWALs0()
final void	init(WALFactory factory, org.apache.hadoop.conf.Configuration conf, String providerId, Abortable server)	Set up the provider to create wals.
protected final void	initWAL(WAL wal)
void	peerSyncReplicationStateChange(String peerId, SyncReplicationState from, SyncReplicationState to, int stage)
private Stream<AbstractFSWAL<?>>	remoteWALStream()
private static void	safeClose(WAL wal)
void	setSyncReplicationPeerInfoProvider(SyncReplicationPeerInfoProvider provider)
final void	shutdown()	persist outstanding WALs to storage and stop accepting new appends.
protected abstract void	shutdown0()

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface org.apache.hadoop.hbase.wal.WALProvider

getWALFileLengthProvider

Field Details

LOG

private static final org.slf4j.Logger LOG

WAL_FILE_NAME_DELIMITER

public static final String WAL_FILE_NAME_DELIMITER

See Also:

• Constant Field Values

factory

protected WALFactory factory

conf

protected org.apache.hadoop.conf.Configuration conf

listeners

protected List<WALActionsListener> listeners

providerId

protected String providerId

initialized

protected AtomicBoolean initialized

logPrefix

protected String logPrefix

abortable

protected Abortable abortable

peerId2WAL

private final ConcurrentMap<String,Optional<WAL>> peerId2WAL

createLock

private final KeyLocker<String> createLock

numRemoteWALUnderCreationLock

private final Lock numRemoteWALUnderCreationLock

noRemoteWALUnderCreationCond

private final Condition noRemoteWALUnderCreationCond

numRemoteWALUnderCreation

private int numRemoteWALUnderCreation

peerInfoProvider

private SyncReplicationPeerInfoProvider peerInfoProvider

LOG_PREFIX_PATTERN

private static final Pattern LOG_PREFIX_PATTERN

Constructor Details

AbstractWALProvider

public AbstractWALProvider()

Method Details

init

public final void init(WALFactory factory,
 org.apache.hadoop.conf.Configuration conf,
 String providerId,
 Abortable server)
                throws IOException

Description copied from interface: WALProvider

Set up the provider to create wals. will only be called once per instance.

Specified by:

init in interface WALProvider

Parameters:

factory - factory that made us may not be null

conf - may not be null

providerId - differentiate between providers from one factory. may be null

Throws:

IOException

initWAL

protected final void initWAL(WAL wal)
                      throws IOException

Throws:

IOException

getRemoteWALPrefix

private String getRemoteWALPrefix(String peerId)

getRemoteWAL

private WAL getRemoteWAL(RegionInfo region,
 String peerId,
 String remoteWALDir)
                  throws IOException

Throws:

IOException

getWAL

public final WAL getWAL(RegionInfo region)
                 throws IOException

Specified by:

getWAL in interface WALProvider

Parameters:

region - the region which we want to get a WAL for it. Could be null.

Returns:

a WAL for writing entries for the given region.

Throws:

IOException

getWALs

public final List<WAL> getWALs()

Description copied from interface: WALProvider

Returns the List of WALs that are used by this server

Specified by:

getWALs in interface WALProvider

getPeerActionListener

public PeerActionListener getPeerActionListener()

Specified by:

getPeerActionListener in interface WALProvider

peerSyncReplicationStateChange

public void peerSyncReplicationStateChange(String peerId,
 SyncReplicationState from,
 SyncReplicationState to,
 int stage)

Specified by:

peerSyncReplicationStateChange in interface PeerActionListener

setSyncReplicationPeerInfoProvider

public void setSyncReplicationPeerInfoProvider(SyncReplicationPeerInfoProvider provider)

Specified by:

setSyncReplicationPeerInfoProvider in interface WALProvider

safeClose

private static void safeClose(WAL wal)

addWALActionsListener

public void addWALActionsListener(WALActionsListener listener)

Description copied from interface: WALProvider

Add a WALActionsListener.

Notice that you must call this method before calling WALProvider.getWAL(RegionInfo) as this method will not effect the WAL which has already been created. And as long as we can only it when initialization, it is not thread safe.

Specified by:

addWALActionsListener in interface WALProvider

cleanup

private void cleanup(IOExceptionConsumer<WAL> cleanupWAL,
 IOExceptionRunnable finalCleanup)
              throws IOException

Throws:

IOException

shutdown

public final void shutdown()
                    throws IOException

Description copied from interface: WALProvider

persist outstanding WALs to storage and stop accepting new appends. This method serves as shorthand for sending a sync to every WAL provided by a given implementation. Those WALs will also stop accepting new writes.

Specified by:

shutdown in interface WALProvider

Throws:

IOException

close

public final void close()
                 throws IOException

Description copied from interface: WALProvider

shutdown utstanding WALs and clean up any persisted state. Call this method only when you will not need to replay any of the edits to the WALs from this provider. After this call completes, the underlying resources should have been reclaimed.

Specified by:

close in interface WALProvider

Throws:

IOException

remoteWALStream

private Stream<AbstractFSWAL<?>> remoteWALStream()

getNumLogFiles

public final long getNumLogFiles()

Description copied from interface: WALProvider

Get number of the log files this provider is managing

Specified by:

getNumLogFiles in interface WALProvider

getLogFileSize

public final long getLogFileSize()

Description copied from interface: WALProvider

Get size of the log files this provider is managing

Specified by:

getLogFileSize in interface WALProvider

getSyncReplicationPeerIdFromWALName

public static Optional<String> getSyncReplicationPeerIdFromWALName(String name)

Returns the peer id if the wal file name is in the special group for a sync replication peer.

The prefix format is <factoryId>-<ts>-<peerId>.

createRemoteWAL

protected abstract WAL createRemoteWAL(RegionInfo region,
 org.apache.hadoop.fs.FileSystem remoteFs,
 org.apache.hadoop.fs.Path remoteWALDir,
 String prefix,
 String suffix)
                                throws IOException

Throws:

IOException

doInit

protected abstract void doInit(WALFactory factory,
 org.apache.hadoop.conf.Configuration conf,
 String providerId)
                        throws IOException

Throws:

IOException

getWAL0

protected abstract WAL getWAL0(RegionInfo region)
                        throws IOException

Throws:

IOException

getWALs0

protected abstract List<WAL> getWALs0()

shutdown0

protected abstract void shutdown0()
                           throws IOException

Throws:

IOException

close0

protected abstract void close0()
                        throws IOException

Throws:

IOException

getNumLogFiles0

protected abstract long getNumLogFiles0()

getLogFileSize0

protected abstract long getLogFileSize0()