@InterfaceAudience.Private @InterfaceStability.Evolving public class BufferedMutatorImpl extends Object implements BufferedMutator
Used to communicate with a single HBase table similar to Table
but meant for batched,
potentially asynchronous puts. Obtain an instance from a Connection
and call
close()
afterwards. Provide an alternate to this implementation by setting
BufferedMutatorParams.implementationClassName(String)
or by setting alternate classname
via the key {} in Configuration.
While this can be used across threads, great care should be used when doing so. Errors are global to the buffered mutator and the Exceptions can be thrown on any thread that causes the flush for requests.
ConnectionFactory
,
Connection
Modifier and Type | Class and Description |
---|---|
(package private) class |
BufferedMutatorImpl.QueueRowAccess |
BufferedMutator.ExceptionListener
Modifier and Type | Field and Description |
---|---|
private AsyncProcess |
ap |
private boolean |
cleanupPoolOnClose |
private boolean |
closed |
private org.apache.hadoop.conf.Configuration |
conf |
private AtomicLong |
currentWriteBufferSize |
private AtomicLong |
executedWriteBufferPeriodicFlushes |
private AtomicLong |
firstRecordInBufferTimestamp |
private BufferedMutator.ExceptionListener |
listener |
private static org.slf4j.Logger |
LOG |
private int |
maxKeyValueSize |
private AtomicInteger |
operationTimeout |
private ExecutorService |
pool |
private AtomicInteger |
rpcTimeout |
private TableName |
tableName |
private AtomicInteger |
undealtMutationCount
Count the size of
writeAsyncBuffer . |
private ConcurrentLinkedQueue<Mutation> |
writeAsyncBuffer |
private AtomicLong |
writeBufferPeriodicFlushTimeoutMs |
private Timer |
writeBufferPeriodicFlushTimer |
private AtomicLong |
writeBufferPeriodicFlushTimerTickMs |
private long |
writeBufferSize |
CLASSNAME_KEY, MIN_WRITE_BUFFER_PERIODIC_FLUSH_TIMERTICK_MS
Constructor and Description |
---|
BufferedMutatorImpl(ClusterConnection conn,
BufferedMutatorParams params,
AsyncProcess ap) |
BufferedMutatorImpl(ClusterConnection conn,
RpcRetryingCallerFactory rpcCallerFactory,
RpcControllerFactory rpcFactory,
BufferedMutatorParams params) |
Modifier and Type | Method and Description |
---|---|
private void |
checkClose() |
void |
close()
Performs a
BufferedMutator.flush() and releases any resources held. |
(package private) BufferedMutatorImpl.QueueRowAccess |
createQueueRowAccess() |
private AsyncProcessTask |
createTask(BufferedMutatorImpl.QueueRowAccess access) |
private void |
doFlush(boolean flushAll)
Send the operations in the buffer to the servers.
|
void |
flush()
Executes all the buffered, asynchronous
Mutation operations and waits until they are
done. |
(package private) AsyncProcess |
getAsyncProcess() |
org.apache.hadoop.conf.Configuration |
getConfiguration()
Returns the
Configuration object used by this instance. |
(package private) long |
getCurrentWriteBufferSize() |
protected long |
getExecutedWriteBufferPeriodicFlushes() |
TableName |
getName()
Gets the fully qualified table name instance of the table that this BufferedMutator writes to.
|
(package private) ExecutorService |
getPool() |
(package private) int |
getUnflushedSize()
Count the mutations which haven't been flushed
|
long |
getWriteBufferPeriodicFlushTimeoutMs()
Returns the current periodic flush timeout value in milliseconds.
|
long |
getWriteBufferPeriodicFlushTimerTickMs()
Returns the current periodic flush timertick interval in milliseconds.
|
long |
getWriteBufferSize()
Returns the maximum size in bytes of the write buffer for this HTable.
|
private static RetriesExhaustedWithDetailsException |
makeException(List<RetriesExhaustedWithDetailsException> errors) |
void |
mutate(List<? extends Mutation> ms)
Send some
Mutation s to the table. |
void |
mutate(Mutation m)
Sends a
Mutation to the table. |
void |
setOperationTimeout(int operationTimeout)
Set operation timeout for this mutator instance
|
void |
setRpcTimeout(int rpcTimeout)
Set rpc timeout for this mutator instance
|
void |
setWriteBufferPeriodicFlush(long timeoutMs,
long timerTickMs)
Sets the maximum time before the buffer is automatically flushed.
|
(package private) int |
size()
Count the mutations which haven't been processed.
|
private void |
timerCallbackForWriteBufferPeriodicFlush() |
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait
disableWriteBufferPeriodicFlush, setWriteBufferPeriodicFlush
private static final org.slf4j.Logger LOG
private final BufferedMutator.ExceptionListener listener
private final org.apache.hadoop.conf.Configuration conf
private final ConcurrentLinkedQueue<Mutation> writeAsyncBuffer
private final AtomicLong currentWriteBufferSize
private final AtomicInteger undealtMutationCount
writeAsyncBuffer
. The
ConcurrentLinkedQueue.size()
is NOT a constant-time operation.private final long writeBufferSize
private final AtomicLong writeBufferPeriodicFlushTimeoutMs
private final AtomicLong writeBufferPeriodicFlushTimerTickMs
private Timer writeBufferPeriodicFlushTimer
private final int maxKeyValueSize
private final ExecutorService pool
private final AtomicInteger rpcTimeout
private final AtomicInteger operationTimeout
private final boolean cleanupPoolOnClose
private volatile boolean closed
private final AsyncProcess ap
private final AtomicLong firstRecordInBufferTimestamp
private final AtomicLong executedWriteBufferPeriodicFlushes
BufferedMutatorImpl(ClusterConnection conn, BufferedMutatorParams params, AsyncProcess ap)
BufferedMutatorImpl(ClusterConnection conn, RpcRetryingCallerFactory rpcCallerFactory, RpcControllerFactory rpcFactory, BufferedMutatorParams params)
private void checkClose()
ExecutorService getPool()
AsyncProcess getAsyncProcess()
public TableName getName()
BufferedMutator
getName
in interface BufferedMutator
public org.apache.hadoop.conf.Configuration getConfiguration()
BufferedMutator
Configuration
object used by this instance.
The reference returned is not a copy, so any change made to it will affect this instance.
getConfiguration
in interface BufferedMutator
public void mutate(Mutation m) throws InterruptedIOException, RetriesExhaustedWithDetailsException
BufferedMutator
Mutation
to the table. The mutations will be buffered and sent over the wire as
part of a batch. Currently only supports Put
and Delete
mutations.mutate
in interface BufferedMutator
m
- The data to send.InterruptedIOException
RetriesExhaustedWithDetailsException
public void mutate(List<? extends Mutation> ms) throws InterruptedIOException, RetriesExhaustedWithDetailsException
BufferedMutator
Mutation
s to the table. The mutations will be buffered and sent over the wire
as part of a batch. There is no guarantee of sending entire content of mutations
in a
single batch; it will be broken up according to the write buffer capacity.mutate
in interface BufferedMutator
ms
- The data to send.InterruptedIOException
RetriesExhaustedWithDetailsException
protected long getExecutedWriteBufferPeriodicFlushes()
private void timerCallbackForWriteBufferPeriodicFlush()
public void close() throws IOException
BufferedMutator
BufferedMutator.flush()
and releases any resources held.close
in interface Closeable
close
in interface AutoCloseable
close
in interface BufferedMutator
IOException
- if a remote or network exception occurs.private AsyncProcessTask createTask(BufferedMutatorImpl.QueueRowAccess access)
public void flush() throws InterruptedIOException, RetriesExhaustedWithDetailsException
BufferedMutator
Mutation
operations and waits until they are
done.flush
in interface BufferedMutator
InterruptedIOException
RetriesExhaustedWithDetailsException
private void doFlush(boolean flushAll) throws InterruptedIOException, RetriesExhaustedWithDetailsException
flushAll
- - if true, sends all the writes and wait for all of them to finish before
returning. Otherwise, flush until buffer size is smaller than thresholdInterruptedIOException
RetriesExhaustedWithDetailsException
private static RetriesExhaustedWithDetailsException makeException(List<RetriesExhaustedWithDetailsException> errors)
public long getWriteBufferSize()
The default value comes from the configuration parameter hbase.client.write.buffer
.
getWriteBufferSize
in interface BufferedMutator
public void setWriteBufferPeriodicFlush(long timeoutMs, long timerTickMs)
BufferedMutator
setWriteBufferPeriodicFlush
in interface BufferedMutator
timeoutMs
- The maximum number of milliseconds how long records may be buffered before
they are flushed. Set to 0 to disable.timerTickMs
- The number of milliseconds between each check if the timeout has been
exceeded. Must be 100ms (as defined in
BufferedMutator.MIN_WRITE_BUFFER_PERIODIC_FLUSH_TIMERTICK_MS
) or larger to avoid
performance problems.public long getWriteBufferPeriodicFlushTimeoutMs()
BufferedMutator
getWriteBufferPeriodicFlushTimeoutMs
in interface BufferedMutator
public long getWriteBufferPeriodicFlushTimerTickMs()
BufferedMutator
getWriteBufferPeriodicFlushTimerTickMs
in interface BufferedMutator
public void setRpcTimeout(int rpcTimeout)
BufferedMutator
setRpcTimeout
in interface BufferedMutator
public void setOperationTimeout(int operationTimeout)
BufferedMutator
setOperationTimeout
in interface BufferedMutator
long getCurrentWriteBufferSize()
int size()
int getUnflushedSize()
BufferedMutatorImpl.QueueRowAccess createQueueRowAccess()
Copyright © 2007–2020 The Apache Software Foundation. All rights reserved.