@InterfaceAudience.Private @InterfaceStability.Evolving public class BufferedMutatorImpl extends Object implements BufferedMutator
Used to communicate with a single HBase table similar to Table
but meant for batched, potentially asynchronous puts. Obtain an instance from
a Connection and call close() afterwards. Provide an alternate
to this implementation by setting BufferedMutatorParams.implementationClassName(String)
or by setting alternate classname via the key {} in Configuration.
While this can be used across threads, great care should be used when doing so. Errors are global to the buffered mutator and the Exceptions can be thrown on any thread that causes the flush for requests.
ConnectionFactory,
Connection| Modifier and Type | Class and Description |
|---|---|
(package private) class |
BufferedMutatorImpl.QueueRowAccess |
BufferedMutator.ExceptionListener| Modifier and Type | Field and Description |
|---|---|
private AsyncProcess |
ap |
private boolean |
cleanupPoolOnClose |
private boolean |
closed |
private org.apache.hadoop.conf.Configuration |
conf |
private AtomicLong |
currentWriteBufferSize |
private AtomicLong |
executedWriteBufferPeriodicFlushes |
private AtomicLong |
firstRecordInBufferTimestamp |
private BufferedMutator.ExceptionListener |
listener |
private static org.slf4j.Logger |
LOG |
private int |
maxKeyValueSize |
private AtomicInteger |
operationTimeout |
private ExecutorService |
pool |
private AtomicInteger |
rpcTimeout |
private TableName |
tableName |
private AtomicInteger |
undealtMutationCount
Count the size of
writeAsyncBuffer. |
private ConcurrentLinkedQueue<Mutation> |
writeAsyncBuffer |
private AtomicLong |
writeBufferPeriodicFlushTimeoutMs |
private Timer |
writeBufferPeriodicFlushTimer |
private AtomicLong |
writeBufferPeriodicFlushTimerTickMs |
private long |
writeBufferSize |
CLASSNAME_KEY, MIN_WRITE_BUFFER_PERIODIC_FLUSH_TIMERTICK_MS| Constructor and Description |
|---|
BufferedMutatorImpl(ClusterConnection conn,
BufferedMutatorParams params,
AsyncProcess ap) |
BufferedMutatorImpl(ClusterConnection conn,
RpcRetryingCallerFactory rpcCallerFactory,
RpcControllerFactory rpcFactory,
BufferedMutatorParams params) |
| Modifier and Type | Method and Description |
|---|---|
private void |
checkClose() |
void |
close()
Performs a
BufferedMutator.flush() and releases any resources held. |
(package private) BufferedMutatorImpl.QueueRowAccess |
createQueueRowAccess() |
private AsyncProcessTask |
createTask(BufferedMutatorImpl.QueueRowAccess access) |
private void |
doFlush(boolean flushAll)
Send the operations in the buffer to the servers.
|
void |
flush()
Executes all the buffered, asynchronous
Mutation operations and waits until they
are done. |
(package private) AsyncProcess |
getAsyncProcess() |
org.apache.hadoop.conf.Configuration |
getConfiguration()
Returns the
Configuration object used by this instance. |
(package private) long |
getCurrentWriteBufferSize() |
protected long |
getExecutedWriteBufferPeriodicFlushes() |
TableName |
getName()
Gets the fully qualified table name instance of the table that this BufferedMutator writes to.
|
(package private) ExecutorService |
getPool() |
(package private) int |
getUnflushedSize()
Count the mutations which haven't been flushed
|
long |
getWriteBufferPeriodicFlushTimeoutMs()
Returns the current periodic flush timeout value in milliseconds.
|
long |
getWriteBufferPeriodicFlushTimerTickMs()
Returns the current periodic flush timertick interval in milliseconds.
|
long |
getWriteBufferSize()
Returns the maximum size in bytes of the write buffer for this HTable.
|
private static RetriesExhaustedWithDetailsException |
makeException(List<RetriesExhaustedWithDetailsException> errors) |
void |
mutate(List<? extends Mutation> ms)
Send some
Mutations to the table. |
void |
mutate(Mutation m)
Sends a
Mutation to the table. |
void |
setOperationTimeout(int operationTimeout)
Set operation timeout for this mutator instance
|
void |
setRpcTimeout(int rpcTimeout)
Set rpc timeout for this mutator instance
|
void |
setWriteBufferPeriodicFlush(long timeoutMs,
long timerTickMs)
Sets the maximum time before the buffer is automatically flushed.
|
(package private) int |
size()
Count the mutations which haven't been processed.
|
private void |
timerCallbackForWriteBufferPeriodicFlush() |
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, waitdisableWriteBufferPeriodicFlush, setWriteBufferPeriodicFlushprivate static final org.slf4j.Logger LOG
private final BufferedMutator.ExceptionListener listener
private final org.apache.hadoop.conf.Configuration conf
private final ConcurrentLinkedQueue<Mutation> writeAsyncBuffer
private final AtomicLong currentWriteBufferSize
private final AtomicInteger undealtMutationCount
writeAsyncBuffer.
The ConcurrentLinkedQueue.size() is NOT a constant-time operation.private final long writeBufferSize
private final AtomicLong writeBufferPeriodicFlushTimeoutMs
private final AtomicLong writeBufferPeriodicFlushTimerTickMs
private Timer writeBufferPeriodicFlushTimer
private final int maxKeyValueSize
private final ExecutorService pool
private final AtomicInteger rpcTimeout
private final AtomicInteger operationTimeout
private final boolean cleanupPoolOnClose
private volatile boolean closed
private final AsyncProcess ap
private final AtomicLong firstRecordInBufferTimestamp
private final AtomicLong executedWriteBufferPeriodicFlushes
BufferedMutatorImpl(ClusterConnection conn, BufferedMutatorParams params, AsyncProcess ap)
BufferedMutatorImpl(ClusterConnection conn, RpcRetryingCallerFactory rpcCallerFactory, RpcControllerFactory rpcFactory, BufferedMutatorParams params)
private void checkClose()
ExecutorService getPool()
AsyncProcess getAsyncProcess()
public TableName getName()
BufferedMutatorgetName in interface BufferedMutatorpublic org.apache.hadoop.conf.Configuration getConfiguration()
BufferedMutatorConfiguration object used by this instance.
The reference returned is not a copy, so any change made to it will affect this instance.
getConfiguration in interface BufferedMutatorpublic void mutate(Mutation m) throws InterruptedIOException, RetriesExhaustedWithDetailsException
BufferedMutatorMutation to the table. The mutations will be buffered and sent over the
wire as part of a batch. Currently only supports Put and Delete mutations.mutate in interface BufferedMutatorm - The data to send.InterruptedIOExceptionRetriesExhaustedWithDetailsExceptionpublic void mutate(List<? extends Mutation> ms) throws InterruptedIOException, RetriesExhaustedWithDetailsException
BufferedMutatorMutations to the table. The mutations will be buffered and sent over the
wire as part of a batch. There is no guarantee of sending entire content of mutations
in a single batch; it will be broken up according to the write buffer capacity.mutate in interface BufferedMutatorms - The data to send.InterruptedIOExceptionRetriesExhaustedWithDetailsExceptionprotected long getExecutedWriteBufferPeriodicFlushes()
private void timerCallbackForWriteBufferPeriodicFlush()
public void close() throws IOException
BufferedMutatorBufferedMutator.flush() and releases any resources held.close in interface Closeableclose in interface AutoCloseableclose in interface BufferedMutatorIOException - if a remote or network exception occurs.private AsyncProcessTask createTask(BufferedMutatorImpl.QueueRowAccess access)
public void flush() throws InterruptedIOException, RetriesExhaustedWithDetailsException
BufferedMutatorMutation operations and waits until they
are done.flush in interface BufferedMutatorInterruptedIOExceptionRetriesExhaustedWithDetailsExceptionprivate void doFlush(boolean flushAll) throws InterruptedIOException, RetriesExhaustedWithDetailsException
flushAll - - if true, sends all the writes and wait for all of them to finish before
returning. Otherwise, flush until buffer size is smaller than thresholdInterruptedIOExceptionRetriesExhaustedWithDetailsExceptionprivate static RetriesExhaustedWithDetailsException makeException(List<RetriesExhaustedWithDetailsException> errors)
public long getWriteBufferSize()
The default value comes from the configuration parameter hbase.client.write.buffer.
getWriteBufferSize in interface BufferedMutatorpublic void setWriteBufferPeriodicFlush(long timeoutMs, long timerTickMs)
BufferedMutatorsetWriteBufferPeriodicFlush in interface BufferedMutatortimeoutMs - The maximum number of milliseconds how long records may be buffered
before they are flushed. Set to 0 to disable.timerTickMs - The number of milliseconds between each check if the
timeout has been exceeded. Must be 100ms (as defined in
BufferedMutator.MIN_WRITE_BUFFER_PERIODIC_FLUSH_TIMERTICK_MS)
or larger to avoid performance problems.public long getWriteBufferPeriodicFlushTimeoutMs()
BufferedMutatorgetWriteBufferPeriodicFlushTimeoutMs in interface BufferedMutatorpublic long getWriteBufferPeriodicFlushTimerTickMs()
BufferedMutatorgetWriteBufferPeriodicFlushTimerTickMs in interface BufferedMutatorpublic void setRpcTimeout(int rpcTimeout)
BufferedMutatorsetRpcTimeout in interface BufferedMutatorpublic void setOperationTimeout(int operationTimeout)
BufferedMutatorsetOperationTimeout in interface BufferedMutatorlong getCurrentWriteBufferSize()
int size()
int getUnflushedSize()
BufferedMutatorImpl.QueueRowAccess createQueueRowAccess()
Copyright © 2007–2019 The Apache Software Foundation. All rights reserved.