org.apache.hadoop.hbase.coprocessor

Class CoprocessorHost<C extends Coprocessor,E extends CoprocessorEnvironment<C>>

java.lang.Object

org.apache.hadoop.hbase.coprocessor.CoprocessorHost<C,E>

Type Parameters:

C - type of specific coprocessor this host will handle

E - type of specific coprocessor environment this host requires. provides

Direct Known Subclasses:

MasterCoprocessorHost, RegionCoprocessorHost, RegionServerCoprocessorHost, WALCoprocessorHost

@InterfaceAudience.Private
public abstract class CoprocessorHost<C extends Coprocessor,E extends CoprocessorEnvironment<C>>
extends Object

Provides the common setup framework and runtime services for coprocessor invocation from HBase services.

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
(package private) static class	CoprocessorHost.EnvironmentPriorityComparator Environment priority comparator.
static interface	CoprocessorHost.ObserverGetter<C,O> Implementations defined function to get an observer of type O from a coprocessor of type C.
private class	CoprocessorHost.ObserverOperation<O>
class	CoprocessorHost.ObserverOperationWithoutResult<O>
class	CoprocessorHost.ObserverOperationWithResult<O,R>

Field Summary

Fields

Modifier and Type	Field and Description
static String	ABORT_ON_ERROR_KEY
protected Abortable	abortable
protected org.apache.hadoop.conf.Configuration	conf
protected SortedList<E>	coprocEnvironments Ordered set of loaded coprocessors with lock
private static Set<String>	coprocessorNames Not to be confused with the per-object _coprocessors_ (above), coprocessorNames is static and stores the set of all coprocessors ever loaded by any thread in this JVM.
static String	COPROCESSORS_ENABLED_CONF_KEY
static boolean	DEFAULT_ABORT_ON_ERROR
static boolean	DEFAULT_COPROCESSORS_ENABLED
static boolean	DEFAULT_SKIP_LOAD_DUPLICATE_TABLE_COPROCESSOR
static boolean	DEFAULT_USER_COPROCESSORS_ENABLED
private static Set<Class<? extends Coprocessor>>	legacyWarning Used to limit legacy handling to once per Coprocessor class per classloader.
protected AtomicInteger	loadSequence
private static org.slf4j.Logger	LOG
static String	MASTER_COPROCESSOR_CONF_KEY
protected String	pathPrefix
static String	REGION_COPROCESSOR_CONF_KEY
static String	REGIONSERVER_COPROCESSOR_CONF_KEY
static String	SKIP_LOAD_DUPLICATE_TABLE_COPROCESSOR
static String	USER_COPROCESSORS_ENABLED_CONF_KEY
static String	USER_REGION_COPROCESSOR_CONF_KEY
static String	WAL_COPROCESSOR_CONF_KEY

Constructor Summary

Constructors

Constructor and Description
CoprocessorHost(Abortable abortable)

Method Summary

Modifier and Type	Method and Description
protected void	abortServer(E environment, Throwable e)
protected void	abortServer(String coprocessorName, Throwable e)
abstract C	checkAndGetInstance(Class<?> implClass) Called when a new Coprocessor class needs to be loaded.
E	checkAndLoadInstance(Class<?> implClass, int priority, org.apache.hadoop.conf.Configuration conf)
abstract E	createEnvironment(C instance, int priority, int sequence, org.apache.hadoop.conf.Configuration conf) Called when a new Coprocessor class is loaded
protected <O> boolean	execOperation(CoprocessorHost.ObserverOperation<O> observerOperation)
protected <O,R> R	execOperationWithResult(CoprocessorHost.ObserverOperationWithResult<O,R> observerOperation) Do not call with an observerOperation that is null! Have the caller check.
protected <O> boolean	execShutdown(CoprocessorHost.ObserverOperation<O> observerOperation) Coprocessor classes can be configured in any order, based on that priority is set and chained in a sorted order.
<T extends C> T	findCoprocessor(Class<T> cls)
C	findCoprocessor(String className) Find coprocessors by full class name or simple name.
E	findCoprocessorEnvironment(String className) Find a coprocessor environment by class name
<T extends C> List<T>	findCoprocessors(Class<T> cls) Find list of coprocessors that extend/implement the given class/interface
Set<String>	getCoprocessors() Used to create a parameter to the HServerLoad constructor so that HServerLoad can provide information about the coprocessors loaded by this regionserver.
(package private) Set<ClassLoader>	getExternalClassLoaders() Retrieves the set of classloaders used to instantiate Coprocessor classes defined in external jar files.
static Set<String>	getLoadedCoprocessors()
protected void	handleCoprocessorThrowable(E env, Throwable e) This is used by coprocessor hooks which are declared to throw IOException (or its subtypes).
void	load(Class<? extends C> implClass, int priority, org.apache.hadoop.conf.Configuration conf)
E	load(org.apache.hadoop.fs.Path path, String className, int priority, org.apache.hadoop.conf.Configuration conf) Load a coprocessor implementation into the host
E	load(org.apache.hadoop.fs.Path path, String className, int priority, org.apache.hadoop.conf.Configuration conf, String[] includedClassPrefixes) Load a coprocessor implementation into the host
protected void	loadSystemCoprocessors(org.apache.hadoop.conf.Configuration conf, String confKey) Load system coprocessors once only.
void	shutdown(E e)

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

REGION_COPROCESSOR_CONF_KEY

public static final String REGION_COPROCESSOR_CONF_KEY

See Also:

Constant Field Values

REGIONSERVER_COPROCESSOR_CONF_KEY

public static final String REGIONSERVER_COPROCESSOR_CONF_KEY

See Also:

Constant Field Values

USER_REGION_COPROCESSOR_CONF_KEY

public static final String USER_REGION_COPROCESSOR_CONF_KEY

See Also:

Constant Field Values

MASTER_COPROCESSOR_CONF_KEY

public static final String MASTER_COPROCESSOR_CONF_KEY

See Also:

Constant Field Values

WAL_COPROCESSOR_CONF_KEY

public static final String WAL_COPROCESSOR_CONF_KEY

See Also:

Constant Field Values

ABORT_ON_ERROR_KEY

public static final String ABORT_ON_ERROR_KEY

See Also:

Constant Field Values

DEFAULT_ABORT_ON_ERROR

public static final boolean DEFAULT_ABORT_ON_ERROR

See Also:

Constant Field Values

COPROCESSORS_ENABLED_CONF_KEY

public static final String COPROCESSORS_ENABLED_CONF_KEY

See Also:

Constant Field Values

DEFAULT_COPROCESSORS_ENABLED

public static final boolean DEFAULT_COPROCESSORS_ENABLED

See Also:

Constant Field Values

USER_COPROCESSORS_ENABLED_CONF_KEY

public static final String USER_COPROCESSORS_ENABLED_CONF_KEY

See Also:

Constant Field Values

DEFAULT_USER_COPROCESSORS_ENABLED

public static final boolean DEFAULT_USER_COPROCESSORS_ENABLED

See Also:

Constant Field Values

SKIP_LOAD_DUPLICATE_TABLE_COPROCESSOR

public static final String SKIP_LOAD_DUPLICATE_TABLE_COPROCESSOR

See Also:

Constant Field Values

DEFAULT_SKIP_LOAD_DUPLICATE_TABLE_COPROCESSOR

public static final boolean DEFAULT_SKIP_LOAD_DUPLICATE_TABLE_COPROCESSOR

See Also:

Constant Field Values

LOG

private static final org.slf4j.Logger LOG

abortable

protected Abortable abortable

coprocEnvironments

protected final SortedList<E extends CoprocessorEnvironment<C>> coprocEnvironments

Ordered set of loaded coprocessors with lock

conf

protected org.apache.hadoop.conf.Configuration conf

pathPrefix

protected String pathPrefix

loadSequence

protected AtomicInteger loadSequence

coprocessorNames

private static Set<String> coprocessorNames

Not to be confused with the per-object _coprocessors_ (above), coprocessorNames is static and stores the set of all coprocessors ever loaded by any thread in this JVM. It is strictly additive: coprocessors are added to coprocessorNames, by checkAndLoadInstance() but are never removed, since the intention is to preserve a history of all loaded coprocessors for diagnosis in case of server crash (HBASE-4014).

legacyWarning

private static final Set<Class<? extends Coprocessor>> legacyWarning

Used to limit legacy handling to once per Coprocessor class per classloader.

Constructor Detail

CoprocessorHost

public CoprocessorHost(Abortable abortable)

Method Detail

getLoadedCoprocessors

public static Set<String> getLoadedCoprocessors()

getCoprocessors

public Set<String> getCoprocessors()

Used to create a parameter to the HServerLoad constructor so that HServerLoad can provide information about the coprocessors loaded by this regionserver. (HBASE-4070: Improve region server metrics to report loaded coprocessors to master).

loadSystemCoprocessors

protected void loadSystemCoprocessors(org.apache.hadoop.conf.Configuration conf,
                                      String confKey)

Load system coprocessors once only. Read the class names from configuration. Called by constructor.

load

public E load(org.apache.hadoop.fs.Path path,
              String className,
              int priority,
              org.apache.hadoop.conf.Configuration conf)
       throws IOException

Load a coprocessor implementation into the host

Parameters:

path - path to implementation jar

className - the main class name

priority - chaining priority

conf - configuration for coprocessor

Throws:

IOException - Exception

load

public E load(org.apache.hadoop.fs.Path path,
              String className,
              int priority,
              org.apache.hadoop.conf.Configuration conf,
              String[] includedClassPrefixes)
       throws IOException

Load a coprocessor implementation into the host

Parameters:

path - path to implementation jar

className - the main class name

priority - chaining priority

conf - configuration for coprocessor

includedClassPrefixes - class name prefixes to include

Throws:

IOException - Exception

load

public void load(Class<? extends C> implClass,
                 int priority,
                 org.apache.hadoop.conf.Configuration conf)
          throws IOException

Throws:

IOException

checkAndLoadInstance

public E checkAndLoadInstance(Class<?> implClass,
                              int priority,
                              org.apache.hadoop.conf.Configuration conf)
                       throws IOException

Parameters:

implClass - Implementation class

priority - priority

conf - configuration

Throws:

IOException - Exception

createEnvironment

public abstract E createEnvironment(C instance,
                                    int priority,
                                    int sequence,
                                    org.apache.hadoop.conf.Configuration conf)

Called when a new Coprocessor class is loaded

checkAndGetInstance

public abstract C checkAndGetInstance(Class<?> implClass)
                               throws InstantiationException,
                                      IllegalAccessException

Called when a new Coprocessor class needs to be loaded. Checks if type of the given class is what the corresponding host implementation expects. If it is of correct type, returns an instance of the coprocessor to be loaded. If not, returns null. If an exception occurs when trying to create instance of a coprocessor, it's passed up and eventually results into server aborting.

Throws:

InstantiationException

IllegalAccessException

shutdown

public void shutdown(E e)

findCoprocessor

public C findCoprocessor(String className)

Find coprocessors by full class name or simple name.

findCoprocessor

public <T extends C> T findCoprocessor(Class<T> cls)

findCoprocessors

public <T extends C> List<T> findCoprocessors(Class<T> cls)

Find list of coprocessors that extend/implement the given class/interface

Parameters:

cls - the class/interface to look for

Returns:

the list of coprocessors, or null if not found

findCoprocessorEnvironment

public E findCoprocessorEnvironment(String className)

Find a coprocessor environment by class name

Parameters:

className - the class name

Returns:

the coprocessor, or null if not found

getExternalClassLoaders

Set<ClassLoader> getExternalClassLoaders()

Retrieves the set of classloaders used to instantiate Coprocessor classes defined in external jar files.

Returns:

A set of ClassLoader instances

abortServer

protected void abortServer(E environment,
                           Throwable e)

abortServer

protected void abortServer(String coprocessorName,
                           Throwable e)

handleCoprocessorThrowable

protected void handleCoprocessorThrowable(E env,
                                          Throwable e)
                                   throws IOException

This is used by coprocessor hooks which are declared to throw IOException (or its subtypes). For such hooks, we should handle throwable objects depending on the Throwable's type. Those which are instances of IOException should be passed on to the client. This is in conformance with the HBase idiom regarding IOException: that it represents a circumstance that should be passed along to the client for its own handling. For example, a coprocessor that implements access controls would throw a subclass of IOException, such as AccessDeniedException, in its preGet() method to prevent an unauthorized client's performing a Get on a particular table.

Parameters:

env - Coprocessor Environment

e - Throwable object thrown by coprocessor.

Throws:

IOException - Exception

execOperationWithResult

protected <O,R> R execOperationWithResult(CoprocessorHost.ObserverOperationWithResult<O,R> observerOperation)
                                   throws IOException

Do not call with an observerOperation that is null! Have the caller check.

Throws:

IOException

execOperation

protected <O> boolean execOperation(CoprocessorHost.ObserverOperation<O> observerOperation)
                             throws IOException

Returns:

True if we are to bypass (Can only be true if ObserverOperation#isBypassable().

Throws:

IOException

execShutdown

protected <O> boolean execShutdown(CoprocessorHost.ObserverOperation<O> observerOperation)
                            throws IOException

Coprocessor classes can be configured in any order, based on that priority is set and chained in a sorted order. Should be used preStop*() hooks i.e. when master/regionserver is going down. This function first calls coprocessor methods (using ObserverOperation.call()) and then shutdowns the environment in postEnvCall().
Need to execute all coprocessor methods first then postEnvCall(), otherwise some coprocessors may remain shutdown if any exception occurs during next coprocessor execution which prevent master/regionserver stop or cluster shutdown. (Refer: HBASE-16663

Returns:

true if bypaas coprocessor execution, false if not.

Throws:

IOException