org.apache.hadoop.hbase.wal

Class IOTestProvider

java.lang.Object

org.apache.hadoop.hbase.wal.IOTestProvider

All Implemented Interfaces:

org.apache.hadoop.hbase.wal.WALProvider

@InterfaceAudience.Private
public class IOTestProvider
extends Object
implements org.apache.hadoop.hbase.wal.WALProvider

A WAL Provider that returns a single thread safe WAL that optionally can skip parts of our normal interactions with HDFS.

This implementation picks a directory in HDFS based on the same mechanisms as the FSHLogProvider. Users can configure how much interaction we have with HDFS with the configuration property "hbase.wal.iotestprovider.operations". The value should be a comma separated list of allowed operations:

• append : edits will be written to the underlying filesystem
• sync : wal syncs will result in hflush calls
• fileroll : roll requests will result in creating a new file on the underlying filesystem.

Additionally, the special cases "all" and "none" are recognized. If ommited, the value defaults to "all." Behavior is undefined if "all" or "none" are paired with additional values. Behavior is also undefined if values not listed above are included.

Only those operations listed will occur between the returned WAL and HDFS. All others will be no-ops.

Note that in the case of allowing "append" operations but not allowing "fileroll", the returned WAL will just keep writing to the same file. This won't avoid all costs associated with file management over time, becaue the data set size may result in additional HDFS block allocations.

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
private static class	IOTestProvider.AllowedOperations
private static class	IOTestProvider.IOTestWAL
private static class	IOTestProvider.IOTestWriter Presumes init will be called by a single thread prior to any access of other methods.

Nested classes/interfaces inherited from interface org.apache.hadoop.hbase.wal.WALProvider

org.apache.hadoop.hbase.wal.WALProvider.AsyncWriter, org.apache.hadoop.hbase.wal.WALProvider.Writer, org.apache.hadoop.hbase.wal.WALProvider.WriterBase

Field Summary

Fields

Modifier and Type	Field and Description
private static String	ALLOWED_OPERATIONS
private org.apache.hadoop.conf.Configuration	conf
private org.apache.hadoop.hbase.wal.WALFactory	factory
protected AtomicBoolean	initialized
private List<org.apache.hadoop.hbase.regionserver.wal.WALActionsListener>	listeners
private org.apache.hadoop.hbase.regionserver.wal.FSHLog	log
private static org.slf4j.Logger	LOG
private String	providerId

Constructor Summary

Constructors

Constructor and Description
IOTestProvider()

Method Summary

Modifier and Type	Method and Description
void	addWALActionsListener(org.apache.hadoop.hbase.regionserver.wal.WALActionsListener listener)
void	close()
private org.apache.hadoop.hbase.regionserver.wal.FSHLog	createWAL()
long	getLogFileSize()
long	getNumLogFiles()
org.apache.hadoop.hbase.wal.WAL	getWAL(org.apache.hadoop.hbase.client.RegionInfo region)
List<org.apache.hadoop.hbase.wal.WAL>	getWALs()
void	init(org.apache.hadoop.hbase.wal.WALFactory factory, org.apache.hadoop.conf.Configuration conf, String providerId, org.apache.hadoop.hbase.Abortable abortable)
void	shutdown()

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface org.apache.hadoop.hbase.wal.WALProvider

getWALFileLengthProvider

Field Detail

LOG

private static final org.slf4j.Logger LOG

ALLOWED_OPERATIONS

private static final String ALLOWED_OPERATIONS

See Also:

Constant Field Values

factory

private org.apache.hadoop.hbase.wal.WALFactory factory

conf

private org.apache.hadoop.conf.Configuration conf

log

private volatile org.apache.hadoop.hbase.regionserver.wal.FSHLog log

providerId

private String providerId

initialized

protected AtomicBoolean initialized

listeners

private List<org.apache.hadoop.hbase.regionserver.wal.WALActionsListener> listeners

Constructor Detail

IOTestProvider

public IOTestProvider()

Method Detail

init

public void init(org.apache.hadoop.hbase.wal.WALFactory factory,
                 org.apache.hadoop.conf.Configuration conf,
                 String providerId,
                 org.apache.hadoop.hbase.Abortable abortable)
          throws IOException

Specified by:

init in interface org.apache.hadoop.hbase.wal.WALProvider

Parameters:

factory - factory that made us, identity used for FS layout. may not be null

conf - may not be null

providerId - differentiate between providers from one facotry, used for FS layout. may be null

Throws:

IOException

getWALs

public List<org.apache.hadoop.hbase.wal.WAL> getWALs()

Specified by:

getWALs in interface org.apache.hadoop.hbase.wal.WALProvider

createWAL

private org.apache.hadoop.hbase.regionserver.wal.FSHLog createWAL()
                                                           throws IOException

Throws:

IOException

getWAL

public org.apache.hadoop.hbase.wal.WAL getWAL(org.apache.hadoop.hbase.client.RegionInfo region)
                                       throws IOException

Specified by:

getWAL in interface org.apache.hadoop.hbase.wal.WALProvider

Throws:

IOException

close

public void close()
           throws IOException

Specified by:

close in interface org.apache.hadoop.hbase.wal.WALProvider

Throws:

IOException

shutdown

public void shutdown()
              throws IOException

Specified by:

shutdown in interface org.apache.hadoop.hbase.wal.WALProvider

Throws:

IOException

getNumLogFiles

public long getNumLogFiles()

Specified by:

getNumLogFiles in interface org.apache.hadoop.hbase.wal.WALProvider

getLogFileSize

public long getLogFileSize()

Specified by:

getLogFileSize in interface org.apache.hadoop.hbase.wal.WALProvider

addWALActionsListener

public void addWALActionsListener(org.apache.hadoop.hbase.regionserver.wal.WALActionsListener listener)

Specified by:

addWALActionsListener in interface org.apache.hadoop.hbase.wal.WALProvider