org.apache.hadoop.hbase.procedure2.store.region

Class RegionProcedureStore

java.lang.Object

org.apache.hadoop.hbase.procedure2.store.ProcedureStoreBase

org.apache.hadoop.hbase.procedure2.store.region.RegionProcedureStore

All Implemented Interfaces:

ProcedureStore

@InterfaceAudience.Private
public class RegionProcedureStore
extends ProcedureStoreBase

A procedure store which uses the master local store to store all the procedures.

We use proc:d column to store the serialized protobuf format procedure, and when deleting we will first fill the info:proc column with an empty byte array, and then actually delete them in the cleanup() method. This is because that we need to retain the max procedure id, so we can not directly delete a procedure row as we do not know if it is the one with the max procedure id.

Nested Class Summary

Nested classes/interfaces inherited from interface org.apache.hadoop.hbase.procedure2.store.ProcedureStore

ProcedureStore.ProcedureIterator, ProcedureStore.ProcedureLoader, ProcedureStore.ProcedureStoreListener

Field Summary

Fields

Modifier and Type	Field and Description
private LeaseRecovery	leaseRecovery
private static org.slf4j.Logger	LOG
private int	numThreads
(package private) static byte[]	PROC_QUALIFIER
(package private) MasterRegion	region
private Server	server
private static org.apache.hbase.thirdparty.com.google.common.collect.ImmutableSet<Class<?>>	UNSUPPORTED_PROCEDURES

Constructor Summary

Constructors

Constructor and Description
RegionProcedureStore(Server server, MasterRegion region, LeaseRecovery leaseRecovery)

Method Summary

Modifier and Type	Method and Description
private void	checkUnsupportedProcedure(Map<Class<?>,List<Procedure<?>>> procsByType) In HBASE-20811, we have introduced a new TRSP to assign/unassign/move regions, and it is incompatible with the old AssignProcedure/UnassignProcedure/MoveRegionProcedure.
void	cleanup() Will be called by the framework to give the store a chance to do some clean up works.
void	delete(long procId) The specified procId was removed from the executor, due to completion, abort or failure.
void	delete(long[] procIds, int offset, int count) The specified procIds were removed from the executor, due to completion, abort or failure.
void	delete(Procedure<?> parentProc, long[] subProcIds) The parent procedure completed.
int	getNumThreads()
void	insert(Procedure<?>[] procs) Serialize a set of new procedures.
void	insert(Procedure<?> proc, Procedure<?>[] subProcs) When a procedure is submitted to the executor insert(proc, null) will be called.
void	load(ProcedureStore.ProcedureLoader loader) Load the Procedures in the store.
void	recoverLease() Acquire the lease for the procedure store.
private void	runWithoutRpcCall(Runnable runnable) Insert procedure may be called by master's rpc call.
private void	serializeDelete(long procId, List<Mutation> mutations, List<byte[]> rowsToLock)
private void	serializePut(Procedure<?> proc, List<Mutation> mutations, List<byte[]> rowsToLock)
int	setRunningProcedureCount(int count) Set the number of procedure running.
void	start(int numThreads) Start/Open the procedure store
void	stop(boolean abort) Stop/Close the procedure store
private void	tryMigrate(org.apache.hadoop.fs.FileSystem fs)
void	update(Procedure<?> proc) The specified procedure was executed, and the new state should be written to the store.

Methods inherited from class org.apache.hadoop.hbase.procedure2.store.ProcedureStoreBase

isRunning, registerListener, sendAbortProcessSignal, sendForceUpdateSignal, sendPostSyncSignal, setRunning, unregisterListener

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

LOG

private static final org.slf4j.Logger LOG

PROC_QUALIFIER

static final byte[] PROC_QUALIFIER

server

private final Server server

leaseRecovery

private final LeaseRecovery leaseRecovery

region

final MasterRegion region

numThreads

private int numThreads

UNSUPPORTED_PROCEDURES

private static final org.apache.hbase.thirdparty.com.google.common.collect.ImmutableSet<Class<?>> UNSUPPORTED_PROCEDURES

Constructor Detail

RegionProcedureStore

public RegionProcedureStore(Server server,
                            MasterRegion region,
                            LeaseRecovery leaseRecovery)

Method Detail

start

public void start(int numThreads)
           throws IOException

Description copied from interface: ProcedureStore

Start/Open the procedure store

Parameters:

numThreads - number of threads to be used by the procedure store

Throws:

IOException

stop

public void stop(boolean abort)

Description copied from interface: ProcedureStore

Stop/Close the procedure store

Parameters:

abort - true if the stop is an abort

getNumThreads

public int getNumThreads()

Returns:

the number of threads/slots passed to start()

setRunningProcedureCount

public int setRunningProcedureCount(int count)

Description copied from interface: ProcedureStore

Set the number of procedure running. This can be used, for example, by the store to know how long to wait before a sync.

Returns:

how many procedures are running (may not be same as count).

checkUnsupportedProcedure

private void checkUnsupportedProcedure(Map<Class<?>,List<Procedure<?>>> procsByType)
                                throws HBaseIOException

In HBASE-20811, we have introduced a new TRSP to assign/unassign/move regions, and it is incompatible with the old AssignProcedure/UnassignProcedure/MoveRegionProcedure. So we need to make sure that there are none these procedures when upgrading. If there are, the master will quit, you need to go back to the old version to finish these procedures first before upgrading.

Throws:

HBaseIOException

tryMigrate

private void tryMigrate(org.apache.hadoop.fs.FileSystem fs)
                 throws IOException

Throws:

IOException

recoverLease

public void recoverLease()
                  throws IOException

Description copied from interface: ProcedureStore

Acquire the lease for the procedure store.

Throws:

IOException

load

public void load(ProcedureStore.ProcedureLoader loader)
          throws IOException

Description copied from interface: ProcedureStore

Load the Procedures in the store.

Parameters:

loader - the ProcedureLoader that will handle the store-load events

Throws:

IOException

serializePut

private void serializePut(Procedure<?> proc,
                          List<Mutation> mutations,
                          List<byte[]> rowsToLock)
                   throws IOException

Throws:

IOException

serializeDelete

private void serializeDelete(long procId,
                             List<Mutation> mutations,
                             List<byte[]> rowsToLock)

runWithoutRpcCall

private void runWithoutRpcCall(Runnable runnable)

Insert procedure may be called by master's rpc call. There are some check about the rpc call when mutate region. Here unset the current rpc call and set it back in finally block. See HBASE-23895 for more details.

insert

public void insert(Procedure<?> proc,
                   Procedure<?>[] subProcs)

Description copied from interface: ProcedureStore

When a procedure is submitted to the executor insert(proc, null) will be called. 'proc' has a 'RUNNABLE' state and the initial information required to start up. When a procedure is executed and it returns children insert(proc, subprocs) will be called. 'proc' has a 'WAITING' state and an update state. 'subprocs' are the children in 'RUNNABLE' state with the initial information.

Parameters:

proc - the procedure to serialize and write to the store.

subProcs - the newly created child of the proc.

insert

public void insert(Procedure<?>[] procs)

Description copied from interface: ProcedureStore

Serialize a set of new procedures. These procedures are freshly submitted to the executor and each procedure has a 'RUNNABLE' state and the initial information required to start up.

Parameters:

procs - the procedures to serialize and write to the store.

update

public void update(Procedure<?> proc)

Description copied from interface: ProcedureStore

The specified procedure was executed, and the new state should be written to the store.

Parameters:

proc - the procedure to serialize and write to the store.

delete

public void delete(long procId)

Description copied from interface: ProcedureStore

The specified procId was removed from the executor, due to completion, abort or failure. The store implementor should remove all the information about the specified procId.

Parameters:

procId - the ID of the procedure to remove.

delete

public void delete(Procedure<?> parentProc,
                   long[] subProcIds)

Description copied from interface: ProcedureStore

The parent procedure completed. Update the state and mark all the child deleted.

Parameters:

parentProc - the parent procedure to serialize and write to the store.

subProcIds - the IDs of the sub-procedure to remove.

delete

public void delete(long[] procIds,
                   int offset,
                   int count)

Description copied from interface: ProcedureStore

The specified procIds were removed from the executor, due to completion, abort or failure. The store implementor should remove all the information about the specified procIds.

Parameters:

procIds - the IDs of the procedures to remove.

offset - the array offset from where to start to delete

count - the number of IDs to delete

cleanup

public void cleanup()

Description copied from interface: ProcedureStore

Will be called by the framework to give the store a chance to do some clean up works.

Notice that this is for periodical clean up work, not for the clean up after close, if you want to close the store just call the ProcedureStore.stop(boolean) method above.