Package org.apache.hadoop.hbase.regionserver

Class HRegion.MutationBatchOperation

java.lang.Object

org.apache.hadoop.hbase.regionserver.HRegion.BatchOperation<Mutation>

org.apache.hadoop.hbase.regionserver.HRegion.MutationBatchOperation

Enclosing class:

HRegion

private static class HRegion.MutationBatchOperation
extends HRegion.BatchOperation<Mutation>

Batch of mutation operations. Base class is shared with HRegion.ReplayBatchOperation as most of the logic is same.

Nested Class Summary

Nested classes/interfaces inherited from class org.apache.hadoop.hbase.regionserver.HRegion.BatchOperation

HRegion.BatchOperation.Visitor

Field Summary

Fields

Modifier and Type	Field	Description
protected boolean	canProceed
private long	nonce
private long	nonceGroup
private boolean	regionReplicateEnable

Fields inherited from class org.apache.hadoop.hbase.regionserver.HRegion.BatchOperation

atomic, durability, familyCellMaps, nextIndexToProcess, observedExceptions, operations, region, results, retCodeDetails, walEditsFromCoprocessors

Constructor Summary

Constructors

Constructor	Description
MutationBatchOperation(HRegion region, Mutation[] operations, boolean atomic, long nonceGroup, long nonce)

Method Summary

Modifier and Type	Method	Description
protected void	addNonSkipWALMutationsToWALEdit(MiniBatchOperationInProgress<Mutation> miniBatchOp, WALEdit walEdit, List<Cell> cellsFromCP, Map<byte[],List<ExtendedCell>> familyCellMap)
private void	attachRegionReplicationToMVCCEntry(MiniBatchOperationInProgress<Mutation> miniBatchOp, MultiVersionConcurrencyControl.WriteEntry mvccWriteEntry, long now)	Create WALKeyImpl and get WALEdit from miniBatchOp and attach RegionReplicationSink.add(org.apache.hadoop.hbase.wal.WALKeyImpl, org.apache.hadoop.hbase.wal.WALEdit, org.apache.hadoop.hbase.ipc.ServerCall<?>) to the mvccWriteEntry.
List<Pair<NonceKey,WALEdit>>	buildWALEdits(MiniBatchOperationInProgress<Mutation> miniBatchOp)	Builds separate WALEdit per nonce by applying input mutations.
protected void	cacheSkipWALMutationForRegionReplication(MiniBatchOperationInProgress<Mutation> miniBatchOp, List<Pair<NonceKey,WALEdit>> nonceKeyAndWALEdits, Map<byte[],List<ExtendedCell>> familyCellMap)	Here is for HBASE-26993,in order to make the new framework for region replication could work for SKIP_WAL, we save the Mutation which Mutation.getDurability() is Durability.SKIP_WAL in miniBatchOp.
private void	callPreMutateCPHook(int index, WALEdit walEdit, int[] metrics)	Runs prePut/preDelete/preIncrement/preAppend coprocessor hook for input mutation in a batch
private void	checkAndMergeCPMutations(MiniBatchOperationInProgress<Mutation> miniBatchOp, List<Region.RowLock> acquiredRowLocks, long timestamp)
void	checkAndPrepare()	Validates each mutation and prepares a batch for write.
void	checkAndPreparePut(Put p)	Implement any Put request specific check and prepare logic here.
void	closeRegionOperation()
void	completeMiniBatchOperations(MiniBatchOperationInProgress<Mutation> miniBatchOp, MultiVersionConcurrencyControl.WriteEntry writeEntry)	This method completes mini-batch operations by calling postBatchMutate() CP hook (if required) and completing mvcc.
private WALEdit	createWALEditForReplicateSkipWAL(MiniBatchOperationInProgress<Mutation> miniBatchOp, List<Pair<NonceKey,WALEdit>> nonceKeyAndWALEdits)
private WALKeyImpl	createWALKey(long now)
void	doPostOpCleanupForMiniBatch(MiniBatchOperationInProgress<Mutation> miniBatchOp, WALEdit walEdit, boolean success)
private void	endNonceOperation(boolean success)	Ends nonce operation for a mutation, if needed.
private static long	getLongValue(Cell cell)	Returns Get the long out of the passed in Cell
Mutation	getMutation(int index)
Mutation[]	getMutationsForCoprocs()	This method is potentially expensive and useful mostly for non-replay CP path.
long	getNonce(int index)
long	getNonceGroup(int index)
long	getOrigLogSeqNum()
boolean	isInReplay()
private void	mergeFamilyMaps(Map<byte[],List<ExtendedCell>> familyMap, Map<byte[],List<ExtendedCell>> toBeMerged)
void	prepareMiniBatchOperations(MiniBatchOperationInProgress<Mutation> miniBatchOp, long timestamp, List<Region.RowLock> acquiredRowLocks)	If necessary, calls preBatchMutate() CP hook for a mini-batch and updates metrics, cell count, tags and timestamp for all cells of all operations in a mini-batch.
private static ExtendedCell	reckonDelta(ExtendedCell delta, ExtendedCell currentCell, byte[] columnFamily, long now, Mutation mutation, Function<ExtendedCell,byte[]> supplier)
private Map<byte[],List<ExtendedCell>>	reckonDeltas(Mutation mutation, List<ExtendedCell> results, long now)
private List<ExtendedCell>	reckonDeltasByStore(HStore store, Mutation mutation, long now, List<ExtendedCell> deltas, List<ExtendedCell> results)	Reckon the Cells to apply to WAL, memstore, and to return to the Client in passed column family/Store.
private boolean	startNonceOperation()	Starts the nonce operation for a mutation, if needed.
void	startRegionOperation()
private static Get	toGet(Mutation mutation)
MultiVersionConcurrencyControl.WriteEntry	writeMiniBatchOperationsToMemStore(MiniBatchOperationInProgress<Mutation> miniBatchOp, MultiVersionConcurrencyControl.WriteEntry writeEntry, long now)	Write mini-batch operations to MemStore

Methods inherited from class org.apache.hadoop.hbase.regionserver.HRegion.BatchOperation

applyFamilyMapToMemStore, checkAndPrepareMutation, checkAndPrepareMutation, createMiniBatch, createWALEdit, doAddCellsToWALEdit, getClusterIds, isAtomic, isDone, isOperationPending, lockRowsAndBuildMiniBatch, size, visitBatchOperations, writeMiniBatchOperationsToMemStore

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Details

nonceGroup

private long nonceGroup

nonce

private long nonce

canProceed

protected boolean canProceed

regionReplicateEnable

private boolean regionReplicateEnable

Constructor Details

MutationBatchOperation

public MutationBatchOperation(HRegion region,
 Mutation[] operations,
 boolean atomic,
 long nonceGroup,
 long nonce)

Method Details

getMutation

public Mutation getMutation(int index)

Specified by:

getMutation in class HRegion.BatchOperation<Mutation>

getNonceGroup

public long getNonceGroup(int index)

Specified by:

getNonceGroup in class HRegion.BatchOperation<Mutation>

getNonce

public long getNonce(int index)

Specified by:

getNonce in class HRegion.BatchOperation<Mutation>

getMutationsForCoprocs

public Mutation[] getMutationsForCoprocs()

Description copied from class: HRegion.BatchOperation

This method is potentially expensive and useful mostly for non-replay CP path.

Specified by:

getMutationsForCoprocs in class HRegion.BatchOperation<Mutation>

isInReplay

public boolean isInReplay()

Specified by:

isInReplay in class HRegion.BatchOperation<Mutation>

getOrigLogSeqNum

public long getOrigLogSeqNum()

Specified by:

getOrigLogSeqNum in class HRegion.BatchOperation<Mutation>

startRegionOperation

public void startRegionOperation()
                          throws IOException

Specified by:

startRegionOperation in class HRegion.BatchOperation<Mutation>

Throws:

IOException

closeRegionOperation

public void closeRegionOperation()
                          throws IOException

Specified by:

closeRegionOperation in class HRegion.BatchOperation<Mutation>

Throws:

IOException

checkAndPreparePut

public void checkAndPreparePut(Put p)
                        throws IOException

Description copied from class: HRegion.BatchOperation

Implement any Put request specific check and prepare logic here. Please refer to HRegion.BatchOperation.checkAndPrepareMutation(Mutation, long) for how its used.

Specified by:

checkAndPreparePut in class HRegion.BatchOperation<Mutation>

Throws:

IOException

checkAndPrepare

public void checkAndPrepare()
                     throws IOException

Description copied from class: HRegion.BatchOperation

Validates each mutation and prepares a batch for write. If necessary (non-replay case), runs CP prePut()/preDelete()/preIncrement()/preAppend() hooks for all mutations in a batch. This is intended to operate on entire batch and will be called from outside of class to check and prepare batch. This can be implemented by calling helper method HRegion.BatchOperation.checkAndPrepareMutation(int, long) in a 'for' loop over mutations.

Specified by:

checkAndPrepare in class HRegion.BatchOperation<Mutation>

Throws:

IOException

prepareMiniBatchOperations

public void prepareMiniBatchOperations(MiniBatchOperationInProgress<Mutation> miniBatchOp,
 long timestamp,
 List<Region.RowLock> acquiredRowLocks)
                                throws IOException

Description copied from class: HRegion.BatchOperation

If necessary, calls preBatchMutate() CP hook for a mini-batch and updates metrics, cell count, tags and timestamp for all cells of all operations in a mini-batch.

Specified by:

prepareMiniBatchOperations in class HRegion.BatchOperation<Mutation>

Throws:

IOException

startNonceOperation

private boolean startNonceOperation()
                             throws IOException

Starts the nonce operation for a mutation, if needed.

Returns:

whether to proceed this mutation.

Throws:

IOException

endNonceOperation

private void endNonceOperation(boolean success)

Ends nonce operation for a mutation, if needed.

Parameters:

success - Whether the operation for this nonce has succeeded.

toGet

private static Get toGet(Mutation mutation)
                  throws IOException

Throws:

IOException

reckonDeltas

private Map<byte[],List<ExtendedCell>> reckonDeltas(Mutation mutation,
 List<ExtendedCell> results,
 long now)
                                             throws IOException

Throws:

IOException

reckonDeltasByStore

private List<ExtendedCell> reckonDeltasByStore(HStore store,
 Mutation mutation,
 long now,
 List<ExtendedCell> deltas,
 List<ExtendedCell> results)
                                        throws IOException

Reckon the Cells to apply to WAL, memstore, and to return to the Client in passed column family/Store. Does Get of current value and then adds passed in deltas for this Store returning the result.

Parameters:

mutation - The encompassing Mutation object

deltas - Changes to apply to this Store; either increment amount or data to append

results - In here we accumulate all the Cells we are to return to the client. If null, client doesn't want results returned.

Returns:

Resulting Cells after deltas have been applied to current values. Side effect is our filling out of the results List.

Throws:

IOException

reckonDelta

private static ExtendedCell reckonDelta(ExtendedCell delta,
 ExtendedCell currentCell,
 byte[] columnFamily,
 long now,
 Mutation mutation,
 Function<ExtendedCell,byte[]> supplier)
                                 throws IOException

Throws:

IOException

getLongValue

private static long getLongValue(Cell cell)
                          throws DoNotRetryIOException

Returns Get the long out of the passed in Cell

Throws:

DoNotRetryIOException

buildWALEdits

public List<Pair<NonceKey,WALEdit>> buildWALEdits(MiniBatchOperationInProgress<Mutation> miniBatchOp)
                                           throws IOException

Description copied from class: HRegion.BatchOperation

Builds separate WALEdit per nonce by applying input mutations. If WALEdits from CP are present, they are merged to result WALEdit.

Overrides:

buildWALEdits in class HRegion.BatchOperation<Mutation>

Throws:

IOException

cacheSkipWALMutationForRegionReplication

protected void cacheSkipWALMutationForRegionReplication(MiniBatchOperationInProgress<Mutation> miniBatchOp,
 List<Pair<NonceKey,WALEdit>> nonceKeyAndWALEdits,
 Map<byte[],List<ExtendedCell>> familyCellMap)

Here is for HBASE-26993,in order to make the new framework for region replication could work for SKIP_WAL, we save the Mutation which Mutation.getDurability() is Durability.SKIP_WAL in miniBatchOp.

Specified by:

cacheSkipWALMutationForRegionReplication in class HRegion.BatchOperation<Mutation>

createWALEditForReplicateSkipWAL

private WALEdit createWALEditForReplicateSkipWAL(MiniBatchOperationInProgress<Mutation> miniBatchOp,
 List<Pair<NonceKey,WALEdit>> nonceKeyAndWALEdits)

addNonSkipWALMutationsToWALEdit

protected void addNonSkipWALMutationsToWALEdit(MiniBatchOperationInProgress<Mutation> miniBatchOp,
 WALEdit walEdit,
 List<Cell> cellsFromCP,
 Map<byte[],List<ExtendedCell>> familyCellMap)

Overrides:

addNonSkipWALMutationsToWALEdit in class HRegion.BatchOperation<Mutation>

writeMiniBatchOperationsToMemStore

public MultiVersionConcurrencyControl.WriteEntry writeMiniBatchOperationsToMemStore(MiniBatchOperationInProgress<Mutation> miniBatchOp,
 @Nullable
 MultiVersionConcurrencyControl.WriteEntry writeEntry,
 long now)
                                                                             throws IOException

Description copied from class: HRegion.BatchOperation

Write mini-batch operations to MemStore

Specified by:

writeMiniBatchOperationsToMemStore in class HRegion.BatchOperation<Mutation>

Throws:

IOException

createWALKey

private WALKeyImpl createWALKey(long now)

attachRegionReplicationToMVCCEntry

private void attachRegionReplicationToMVCCEntry(MiniBatchOperationInProgress<Mutation> miniBatchOp,
 MultiVersionConcurrencyControl.WriteEntry mvccWriteEntry,
 long now)
                                         throws IOException

Create WALKeyImpl and get WALEdit from miniBatchOp and attach RegionReplicationSink.add(org.apache.hadoop.hbase.wal.WALKeyImpl, org.apache.hadoop.hbase.wal.WALEdit, org.apache.hadoop.hbase.ipc.ServerCall<?>) to the mvccWriteEntry.

Throws:

IOException

completeMiniBatchOperations

public void completeMiniBatchOperations(MiniBatchOperationInProgress<Mutation> miniBatchOp,
 MultiVersionConcurrencyControl.WriteEntry writeEntry)
                                 throws IOException

Description copied from class: HRegion.BatchOperation

This method completes mini-batch operations by calling postBatchMutate() CP hook (if required) and completing mvcc.

Overrides:

completeMiniBatchOperations in class HRegion.BatchOperation<Mutation>

Throws:

IOException

doPostOpCleanupForMiniBatch

public void doPostOpCleanupForMiniBatch(MiniBatchOperationInProgress<Mutation> miniBatchOp,
 WALEdit walEdit,
 boolean success)
                                 throws IOException

Overrides:

doPostOpCleanupForMiniBatch in class HRegion.BatchOperation<Mutation>

Throws:

IOException

callPreMutateCPHook

private void callPreMutateCPHook(int index,
 WALEdit walEdit,
 int[] metrics)
                          throws IOException

Runs prePut/preDelete/preIncrement/preAppend coprocessor hook for input mutation in a batch

Parameters:

metrics - Array of 2 ints. index 0: count of puts, index 1: count of deletes, index 2: count of increments and 3: count of appends

Throws:

IOException

checkAndMergeCPMutations

private void checkAndMergeCPMutations(MiniBatchOperationInProgress<Mutation> miniBatchOp,
 List<Region.RowLock> acquiredRowLocks,
 long timestamp)
                               throws IOException

Throws:

IOException

mergeFamilyMaps

private void mergeFamilyMaps(Map<byte[],List<ExtendedCell>> familyMap,
 Map<byte[],List<ExtendedCell>> toBeMerged)