Class HRegion.MutationBatchOperation
java.lang.Object
org.apache.hadoop.hbase.regionserver.HRegion.BatchOperation<Mutation>
org.apache.hadoop.hbase.regionserver.HRegion.MutationBatchOperation
- Enclosing class:
- HRegion
Batch of mutation operations. Base class is shared with
HRegion.ReplayBatchOperation
as most of
the logic is same.-
Nested Class Summary
Nested classes/interfaces inherited from class org.apache.hadoop.hbase.regionserver.HRegion.BatchOperation
HRegion.BatchOperation.Visitor
-
Field Summary
Modifier and TypeFieldDescriptionprotected boolean
private long
private long
private boolean
Fields inherited from class org.apache.hadoop.hbase.regionserver.HRegion.BatchOperation
atomic, durability, familyCellMaps, nextIndexToProcess, observedExceptions, operations, region, results, retCodeDetails, walEditsFromCoprocessors
-
Constructor Summary
ConstructorDescriptionMutationBatchOperation
(HRegion region, Mutation[] operations, boolean atomic, long nonceGroup, long nonce) -
Method Summary
Modifier and TypeMethodDescriptionprotected void
addNonSkipWALMutationsToWALEdit
(MiniBatchOperationInProgress<Mutation> miniBatchOp, WALEdit walEdit, List<ExtendedCell> cellsFromCP, Map<byte[], List<ExtendedCell>> familyCellMap) private void
attachRegionReplicationToMVCCEntry
(MiniBatchOperationInProgress<Mutation> miniBatchOp, MultiVersionConcurrencyControl.WriteEntry mvccWriteEntry, long now) CreateWALKeyImpl
and getWALEdit
from miniBatchOp and attachRegionReplicationSink.add(org.apache.hadoop.hbase.wal.WALKeyImpl, org.apache.hadoop.hbase.wal.WALEdit, org.apache.hadoop.hbase.ipc.ServerCall<?>)
to the mvccWriteEntry.buildWALEdits
(MiniBatchOperationInProgress<Mutation> miniBatchOp) Builds separate WALEdit per nonce by applying input mutations.protected void
cacheSkipWALMutationForRegionReplication
(MiniBatchOperationInProgress<Mutation> miniBatchOp, List<Pair<NonceKey, WALEdit>> nonceKeyAndWALEdits, Map<byte[], List<ExtendedCell>> familyCellMap) Here is for HBASE-26993,in order to make the new framework for region replication could work for SKIP_WAL, we save theMutation
whichMutation.getDurability()
isDurability.SKIP_WAL
in miniBatchOp.private void
callPreMutateCPHook
(int index, WALEdit walEdit, int[] metrics) Runs prePut/preDelete/preIncrement/preAppend coprocessor hook for input mutation in a batchprivate void
checkAndMergeCPMutations
(MiniBatchOperationInProgress<Mutation> miniBatchOp, List<Region.RowLock> acquiredRowLocks, long timestamp) void
Validates each mutation and prepares a batch for write.void
Implement any Put request specific check and prepare logic here.void
void
completeMiniBatchOperations
(MiniBatchOperationInProgress<Mutation> miniBatchOp, MultiVersionConcurrencyControl.WriteEntry writeEntry) This method completes mini-batch operations by calling postBatchMutate() CP hook (if required) and completing mvcc.private WALEdit
createWALEditForReplicateSkipWAL
(MiniBatchOperationInProgress<Mutation> miniBatchOp, List<Pair<NonceKey, WALEdit>> nonceKeyAndWALEdits) private WALKeyImpl
createWALKey
(long now) void
doPostOpCleanupForMiniBatch
(MiniBatchOperationInProgress<Mutation> miniBatchOp, WALEdit walEdit, boolean success) private void
endNonceOperation
(boolean success) Ends nonce operation for a mutation, if needed.private static long
getLongValue
(Cell cell) Returns Get the long out of the passed in CellgetMutation
(int index) Mutation[]
This method is potentially expensive and useful mostly for non-replay CP path.long
getNonce
(int index) long
getNonceGroup
(int index) long
boolean
private void
mergeFamilyMaps
(Map<byte[], List<ExtendedCell>> familyMap, Map<byte[], List<ExtendedCell>> toBeMerged) void
prepareMiniBatchOperations
(MiniBatchOperationInProgress<Mutation> miniBatchOp, long timestamp, List<Region.RowLock> acquiredRowLocks) If necessary, calls preBatchMutate() CP hook for a mini-batch and updates metrics, cell count, tags and timestamp for all cells of all operations in a mini-batch.private static ExtendedCell
reckonDelta
(ExtendedCell delta, ExtendedCell currentCell, byte[] columnFamily, long now, Mutation mutation, Function<ExtendedCell, byte[]> supplier) private Map<byte[],
List<ExtendedCell>> reckonDeltas
(Mutation mutation, List<ExtendedCell> results, long now) private List<ExtendedCell>
reckonDeltasByStore
(HStore store, Mutation mutation, long now, List<ExtendedCell> deltas, List<ExtendedCell> results) Reckon the Cells to apply to WAL, memstore, and to return to the Client in passed column family/Store.private boolean
Starts the nonce operation for a mutation, if needed.void
private static Get
writeMiniBatchOperationsToMemStore
(MiniBatchOperationInProgress<Mutation> miniBatchOp, MultiVersionConcurrencyControl.WriteEntry writeEntry, long now) Write mini-batch operations to MemStoreMethods inherited from class org.apache.hadoop.hbase.regionserver.HRegion.BatchOperation
applyFamilyMapToMemStore, checkAndPrepareMutation, checkAndPrepareMutation, createMiniBatch, createWALEdit, doAddCellsToWALEdit, getClusterIds, isAtomic, isDone, isOperationPending, lockRowsAndBuildMiniBatch, size, visitBatchOperations, writeMiniBatchOperationsToMemStore
-
Field Details
-
nonceGroup
-
nonce
-
canProceed
-
regionReplicateEnable
-
-
Constructor Details
-
MutationBatchOperation
public MutationBatchOperation(HRegion region, Mutation[] operations, boolean atomic, long nonceGroup, long nonce)
-
-
Method Details
-
getMutation
- Specified by:
getMutation
in classHRegion.BatchOperation<Mutation>
-
getNonceGroup
- Specified by:
getNonceGroup
in classHRegion.BatchOperation<Mutation>
-
getNonce
- Specified by:
getNonce
in classHRegion.BatchOperation<Mutation>
-
getMutationsForCoprocs
Description copied from class:HRegion.BatchOperation
This method is potentially expensive and useful mostly for non-replay CP path.- Specified by:
getMutationsForCoprocs
in classHRegion.BatchOperation<Mutation>
-
isInReplay
- Specified by:
isInReplay
in classHRegion.BatchOperation<Mutation>
-
getOrigLogSeqNum
- Specified by:
getOrigLogSeqNum
in classHRegion.BatchOperation<Mutation>
-
startRegionOperation
- Specified by:
startRegionOperation
in classHRegion.BatchOperation<Mutation>
- Throws:
IOException
-
closeRegionOperation
- Specified by:
closeRegionOperation
in classHRegion.BatchOperation<Mutation>
- Throws:
IOException
-
checkAndPreparePut
Description copied from class:HRegion.BatchOperation
Implement any Put request specific check and prepare logic here. Please refer toHRegion.BatchOperation.checkAndPrepareMutation(Mutation, long)
for how its used.- Specified by:
checkAndPreparePut
in classHRegion.BatchOperation<Mutation>
- Throws:
IOException
-
checkAndPrepare
Description copied from class:HRegion.BatchOperation
Validates each mutation and prepares a batch for write. If necessary (non-replay case), runs CP prePut()/preDelete()/preIncrement()/preAppend() hooks for all mutations in a batch. This is intended to operate on entire batch and will be called from outside of class to check and prepare batch. This can be implemented by calling helper methodHRegion.BatchOperation.checkAndPrepareMutation(int, long)
in a 'for' loop over mutations.- Specified by:
checkAndPrepare
in classHRegion.BatchOperation<Mutation>
- Throws:
IOException
-
prepareMiniBatchOperations
public void prepareMiniBatchOperations(MiniBatchOperationInProgress<Mutation> miniBatchOp, long timestamp, List<Region.RowLock> acquiredRowLocks) throws IOException Description copied from class:HRegion.BatchOperation
If necessary, calls preBatchMutate() CP hook for a mini-batch and updates metrics, cell count, tags and timestamp for all cells of all operations in a mini-batch.- Specified by:
prepareMiniBatchOperations
in classHRegion.BatchOperation<Mutation>
- Throws:
IOException
-
startNonceOperation
Starts the nonce operation for a mutation, if needed.- Returns:
- whether to proceed this mutation.
- Throws:
IOException
-
endNonceOperation
Ends nonce operation for a mutation, if needed.- Parameters:
success
- Whether the operation for this nonce has succeeded.
-
toGet
- Throws:
IOException
-
reckonDeltas
private Map<byte[],List<ExtendedCell>> reckonDeltas(Mutation mutation, List<ExtendedCell> results, long now) throws IOException - Throws:
IOException
-
reckonDeltasByStore
private List<ExtendedCell> reckonDeltasByStore(HStore store, Mutation mutation, long now, List<ExtendedCell> deltas, List<ExtendedCell> results) throws IOException Reckon the Cells to apply to WAL, memstore, and to return to the Client in passed column family/Store. Does Get of current value and then adds passed in deltas for this Store returning the result.- Parameters:
mutation
- The encompassing Mutation objectdeltas
- Changes to apply to this Store; either increment amount or data to appendresults
- In here we accumulate all the Cells we are to return to the client. If null, client doesn't want results returned.- Returns:
- Resulting Cells after
deltas
have been applied to current values. Side effect is our filling out of theresults
List. - Throws:
IOException
-
reckonDelta
private static ExtendedCell reckonDelta(ExtendedCell delta, ExtendedCell currentCell, byte[] columnFamily, long now, Mutation mutation, Function<ExtendedCell, byte[]> supplier) throws IOException- Throws:
IOException
-
getLongValue
Returns Get the long out of the passed in Cell- Throws:
DoNotRetryIOException
-
buildWALEdits
public List<Pair<NonceKey,WALEdit>> buildWALEdits(MiniBatchOperationInProgress<Mutation> miniBatchOp) throws IOException Description copied from class:HRegion.BatchOperation
Builds separate WALEdit per nonce by applying input mutations. If WALEdits from CP are present, they are merged to result WALEdit.- Overrides:
buildWALEdits
in classHRegion.BatchOperation<Mutation>
- Throws:
IOException
-
cacheSkipWALMutationForRegionReplication
protected void cacheSkipWALMutationForRegionReplication(MiniBatchOperationInProgress<Mutation> miniBatchOp, List<Pair<NonceKey, WALEdit>> nonceKeyAndWALEdits, Map<byte[], List<ExtendedCell>> familyCellMap) Here is for HBASE-26993,in order to make the new framework for region replication could work for SKIP_WAL, we save theMutation
whichMutation.getDurability()
isDurability.SKIP_WAL
in miniBatchOp.- Specified by:
cacheSkipWALMutationForRegionReplication
in classHRegion.BatchOperation<Mutation>
-
createWALEditForReplicateSkipWAL
private WALEdit createWALEditForReplicateSkipWAL(MiniBatchOperationInProgress<Mutation> miniBatchOp, List<Pair<NonceKey, WALEdit>> nonceKeyAndWALEdits) -
addNonSkipWALMutationsToWALEdit
protected void addNonSkipWALMutationsToWALEdit(MiniBatchOperationInProgress<Mutation> miniBatchOp, WALEdit walEdit, List<ExtendedCell> cellsFromCP, Map<byte[], List<ExtendedCell>> familyCellMap) - Overrides:
addNonSkipWALMutationsToWALEdit
in classHRegion.BatchOperation<Mutation>
-
writeMiniBatchOperationsToMemStore
public MultiVersionConcurrencyControl.WriteEntry writeMiniBatchOperationsToMemStore(MiniBatchOperationInProgress<Mutation> miniBatchOp, @Nullable MultiVersionConcurrencyControl.WriteEntry writeEntry, long now) throws IOException Description copied from class:HRegion.BatchOperation
Write mini-batch operations to MemStore- Specified by:
writeMiniBatchOperationsToMemStore
in classHRegion.BatchOperation<Mutation>
- Throws:
IOException
-
createWALKey
-
attachRegionReplicationToMVCCEntry
private void attachRegionReplicationToMVCCEntry(MiniBatchOperationInProgress<Mutation> miniBatchOp, MultiVersionConcurrencyControl.WriteEntry mvccWriteEntry, long now) throws IOException CreateWALKeyImpl
and getWALEdit
from miniBatchOp and attachRegionReplicationSink.add(org.apache.hadoop.hbase.wal.WALKeyImpl, org.apache.hadoop.hbase.wal.WALEdit, org.apache.hadoop.hbase.ipc.ServerCall<?>)
to the mvccWriteEntry.- Throws:
IOException
-
completeMiniBatchOperations
public void completeMiniBatchOperations(MiniBatchOperationInProgress<Mutation> miniBatchOp, MultiVersionConcurrencyControl.WriteEntry writeEntry) throws IOException Description copied from class:HRegion.BatchOperation
This method completes mini-batch operations by calling postBatchMutate() CP hook (if required) and completing mvcc.- Overrides:
completeMiniBatchOperations
in classHRegion.BatchOperation<Mutation>
- Throws:
IOException
-
doPostOpCleanupForMiniBatch
public void doPostOpCleanupForMiniBatch(MiniBatchOperationInProgress<Mutation> miniBatchOp, WALEdit walEdit, boolean success) throws IOException - Overrides:
doPostOpCleanupForMiniBatch
in classHRegion.BatchOperation<Mutation>
- Throws:
IOException
-
callPreMutateCPHook
Runs prePut/preDelete/preIncrement/preAppend coprocessor hook for input mutation in a batch- Parameters:
metrics
- Array of 2 ints. index 0: count of puts, index 1: count of deletes, index 2: count of increments and 3: count of appends- Throws:
IOException
-
checkAndMergeCPMutations
private void checkAndMergeCPMutations(MiniBatchOperationInProgress<Mutation> miniBatchOp, List<Region.RowLock> acquiredRowLocks, long timestamp) throws IOException - Throws:
IOException
-
mergeFamilyMaps
private void mergeFamilyMaps(Map<byte[], List<ExtendedCell>> familyMap, Map<byte[], List<ExtendedCell>> toBeMerged)
-