org.apache.hadoop.hbase.client

Class HTableMultiplexer

java.lang.Object

org.apache.hadoop.hbase.client.HTableMultiplexer

@InterfaceAudience.Public
public class HTableMultiplexer
extends Object

HTableMultiplexer provides a thread-safe non blocking PUT API across all the tables. Each put will be sharded into different buffer queues based on its destination region server. So each region server buffer queue will only have the puts which share the same destination. And each queue will have a flush worker thread to flush the puts request to the region server. If any queue is full, the HTableMultiplexer starts to drop the Put requests for that particular queue. Also all the puts will be retried as a configuration number before dropping. And the HTableMultiplexer can report the number of buffered requests and the number of the failed (dropped) requests in total or on per region server basis. This class is thread safe.

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
private static class	HTableMultiplexer.AtomicAverageCounter Helper to count the average over an interval until reset.
(package private) static class	HTableMultiplexer.FlushWorker
static class	HTableMultiplexer.HTableMultiplexerStatus HTableMultiplexerStatus keeps track of the current status of the HTableMultiplexer.
(package private) static class	HTableMultiplexer.PutStatus

Field Summary

Fields

Modifier and Type	Field and Description
private ClusterConnection	conn
private ScheduledExecutorService	executor
private long	flushPeriod
private static org.slf4j.Logger	LOG
private int	maxAttempts
private int	maxKeyValueSize
private int	perRegionServerBufferQueueSize
private ExecutorService	pool
private Map<HRegionLocation,HTableMultiplexer.FlushWorker>	serverToFlushWorkerMap The map between each region server to its flush worker
static String	TABLE_MULTIPLEXER_FLUSH_PERIOD_MS
static String	TABLE_MULTIPLEXER_INIT_THREADS
static String	TABLE_MULTIPLEXER_MAX_RETRIES_IN_QUEUE
private org.apache.hadoop.conf.Configuration	workerConf

Constructor Summary

Constructors

Constructor and Description
HTableMultiplexer(org.apache.hadoop.conf.Configuration conf, int perRegionServerBufferQueueSize)
HTableMultiplexer(Connection conn, org.apache.hadoop.conf.Configuration conf, int perRegionServerBufferQueueSize)

Method Summary

Modifier and Type	Method and Description
void	close() Closes the internal Connection.
(package private) ClusterConnection	getConnection()
HTableMultiplexer.HTableMultiplexerStatus	getHTableMultiplexerStatus()
(package private) LinkedBlockingQueue<HTableMultiplexer.PutStatus>	getQueue(HRegionLocation addr)
List<Put>	put(byte[] tableName, List<Put> puts) Deprecated. Use put(TableName, List) instead.
boolean	put(byte[] tableName, Put put) Deprecated. Use put(TableName, Put) instead.
boolean	put(byte[] tableName, Put put, int retry) Deprecated. Use put(TableName, Put) instead.
List<Put>	put(TableName tableName, List<Put> puts) The puts request will be buffered by their corresponding buffer queue.
boolean	put(TableName tableName, Put put) The put request will be buffered by its corresponding buffer queue.
boolean	put(TableName tableName, Put put, int maxAttempts) The put request will be buffered by its corresponding buffer queue.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

LOG

private static final org.slf4j.Logger LOG

TABLE_MULTIPLEXER_FLUSH_PERIOD_MS

public static final String TABLE_MULTIPLEXER_FLUSH_PERIOD_MS

See Also:

Constant Field Values

TABLE_MULTIPLEXER_INIT_THREADS

public static final String TABLE_MULTIPLEXER_INIT_THREADS

See Also:

Constant Field Values

TABLE_MULTIPLEXER_MAX_RETRIES_IN_QUEUE

public static final String TABLE_MULTIPLEXER_MAX_RETRIES_IN_QUEUE

See Also:

Constant Field Values

serverToFlushWorkerMap

private final Map<HRegionLocation,HTableMultiplexer.FlushWorker> serverToFlushWorkerMap

The map between each region server to its flush worker

workerConf

private final org.apache.hadoop.conf.Configuration workerConf

conn

private final ClusterConnection conn

pool

private final ExecutorService pool

maxAttempts

private final int maxAttempts

perRegionServerBufferQueueSize

private final int perRegionServerBufferQueueSize

maxKeyValueSize

private final int maxKeyValueSize

executor

private final ScheduledExecutorService executor

flushPeriod

private final long flushPeriod

Constructor Detail

HTableMultiplexer

public HTableMultiplexer(org.apache.hadoop.conf.Configuration conf,
                         int perRegionServerBufferQueueSize)
                  throws IOException

Parameters:

conf - The HBaseConfiguration

perRegionServerBufferQueueSize - determines the max number of the buffered Put ops for each region server before dropping the request.

Throws:

IOException

HTableMultiplexer

public HTableMultiplexer(Connection conn,
                         org.apache.hadoop.conf.Configuration conf,
                         int perRegionServerBufferQueueSize)

Parameters:

conn - The HBase connection.

conf - The HBase configuration

perRegionServerBufferQueueSize - determines the max number of the buffered Put ops for each region server before dropping the request.

Method Detail

close

public void close()
           throws IOException

Closes the internal Connection. Does nothing if the Connection has already been closed.

Throws:

IOException - If there is an error closing the connection.

put

public boolean put(TableName tableName,
                   Put put)

The put request will be buffered by its corresponding buffer queue. Return false if the queue is already full.

Parameters:

tableName -

put -

Returns:

true if the request can be accepted by its corresponding buffer queue.

put

public List<Put> put(TableName tableName,
                     List<Put> puts)

The puts request will be buffered by their corresponding buffer queue. Return the list of puts which could not be queued.

Parameters:

tableName -

puts -

Returns:

the list of puts which could not be queued

put

@Deprecated
public List<Put> put(byte[] tableName,
                                 List<Put> puts)

Deprecated. Use put(TableName, List) instead.

put

public boolean put(TableName tableName,
                   Put put,
                   int maxAttempts)

The put request will be buffered by its corresponding buffer queue. And the put request will be retried before dropping the request. Return false if the queue is already full.

Returns:

true if the request can be accepted by its corresponding buffer queue.

put

@Deprecated
public boolean put(byte[] tableName,
                               Put put,
                               int retry)

Deprecated. Use put(TableName, Put) instead.

put

@Deprecated
public boolean put(byte[] tableName,
                               Put put)

Deprecated. Use put(TableName, Put) instead.

getHTableMultiplexerStatus

public HTableMultiplexer.HTableMultiplexerStatus getHTableMultiplexerStatus()

Returns:

the current HTableMultiplexerStatus

getQueue

LinkedBlockingQueue<HTableMultiplexer.PutStatus> getQueue(HRegionLocation addr)

getConnection

ClusterConnection getConnection()