org.apache.hadoop.hbase.util

Class CoprocessorClassLoader

java.lang.Object

java.lang.ClassLoader

java.security.SecureClassLoader

java.net.URLClassLoader

org.apache.hadoop.hbase.util.ClassLoaderBase

org.apache.hadoop.hbase.util.CoprocessorClassLoader

All Implemented Interfaces:

Closeable, AutoCloseable

@InterfaceAudience.Private
public class CoprocessorClassLoader
extends ClassLoaderBase

ClassLoader used to load classes for Coprocessor instances.

This ClassLoader always tries to load classes from the specified coprocessor jar first actually using URLClassLoader logic before delegating to the parent ClassLoader, thus avoiding dependency conflicts between HBase's classpath and classes in the coprocessor jar.

Certain classes are exempt from being loaded by this ClassLoader because it would prevent them from being cast to the equivalent classes in the region server. For example, the Coprocessor interface needs to be loaded by the region server's ClassLoader to prevent a ClassCastException when casting the coprocessor implementation.

A HDFS path can be used to specify the coprocessor jar. In this case, the jar will be copied to local at first under some folder under ${hbase.local.dir}/jars/tmp/. The local copy will be removed automatically when the HBase server instance is stopped.

This ClassLoader also handles resource loading. In most cases this ClassLoader will attempt to load resources from the coprocessor jar first before delegating to the parent. However, like in class loading, some resources need to be handled differently. For all of the Hadoop default configurations (e.g. hbase-default.xml) we will check the parent ClassLoader first to prevent issues such as failing the HBase default configuration version check.

Field Summary

Fields

Modifier and Type	Field and Description
private static String[]	CLASS_PREFIX_EXEMPTIONS If the class being loaded starts with any of these strings, we will skip trying to load it from the coprocessor jar and instead delegate directly to the parent ClassLoader.
private static ConcurrentMap<org.apache.hadoop.fs.Path,CoprocessorClassLoader>	classLoadersCache External class loaders cache keyed by external jar path.
private static Pattern	libJarPattern
private static KeyLocker<String>	locker A locker used to synchronize class loader initialization per coprocessor jar file
private static org.slf4j.Logger	LOG
(package private) static HashSet<String>	parentDirLockSet A set used to synchronized parent path clean up.
private static Pattern[]	RESOURCE_LOAD_PARENT_FIRST_PATTERNS If the resource being loaded matches any of these patterns, we will first attempt to load the resource with the parent ClassLoader.
private static String	TMP_JARS_DIR

Fields inherited from class org.apache.hadoop.hbase.util.ClassLoaderBase

DEFAULT_LOCAL_DIR, LOCAL_DIR_KEY, parent

Constructor Summary

Constructors

Modifier	Constructor and Description
private	CoprocessorClassLoader(ClassLoader parent) Creates a JarClassLoader that loads classes from the given paths.

Method Summary

Modifier and Type	Method and Description
static void	clearCache()
static Collection<? extends ClassLoader>	getAllCached()
static CoprocessorClassLoader	getClassLoader(org.apache.hadoop.fs.Path path, ClassLoader parent, String pathPrefix, org.apache.hadoop.conf.Configuration conf) Get a CoprocessorClassLoader for a coprocessor jar path from cache.
static CoprocessorClassLoader	getIfCached(org.apache.hadoop.fs.Path path)
URL	getResource(String name)
private void	init(org.apache.hadoop.fs.Path pathPattern, String pathPrefix, org.apache.hadoop.conf.Configuration conf)
protected boolean	isClassExempt(String name, String[] includedClassPrefixes) Determines whether the given class should be exempt from being loaded by this ClassLoader.
Class<?>	loadClass(String name)
Class<?>	loadClass(String name, String[] includedClassPrefixes)
protected boolean	loadResourceUsingParentFirst(String name) Determines whether we should attempt to load the given resource using the parent first before attempting to load the resource using this ClassLoader.

Methods inherited from class java.net.URLClassLoader

addURL, close, definePackage, findClass, findResource, findResources, getPermissions, getResourceAsStream, getURLs, newInstance, newInstance

Methods inherited from class java.security.SecureClassLoader

defineClass, defineClass

Methods inherited from class java.lang.ClassLoader

clearAssertionStatus, defineClass, defineClass, defineClass, defineClass, definePackage, findLibrary, findLoadedClass, findSystemClass, getClassLoadingLock, getPackage, getPackages, getParent, getResources, getSystemClassLoader, getSystemResource, getSystemResourceAsStream, getSystemResources, loadClass, registerAsParallelCapable, resolveClass, setClassAssertionStatus, setDefaultAssertionStatus, setPackageAssertionStatus, setSigners

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

LOG

private static final org.slf4j.Logger LOG

TMP_JARS_DIR

private static final String TMP_JARS_DIR

classLoadersCache

private static final ConcurrentMap<org.apache.hadoop.fs.Path,CoprocessorClassLoader> classLoadersCache

External class loaders cache keyed by external jar path. ClassLoader instance is stored as a weak-reference to allow GC'ing when it is not used (@see HBASE-7205)

CLASS_PREFIX_EXEMPTIONS

private static final String[] CLASS_PREFIX_EXEMPTIONS

If the class being loaded starts with any of these strings, we will skip trying to load it from the coprocessor jar and instead delegate directly to the parent ClassLoader.

RESOURCE_LOAD_PARENT_FIRST_PATTERNS

private static final Pattern[] RESOURCE_LOAD_PARENT_FIRST_PATTERNS

If the resource being loaded matches any of these patterns, we will first attempt to load the resource with the parent ClassLoader. Only if the resource is not found by the parent do we attempt to load it from the coprocessor jar.

libJarPattern

private static final Pattern libJarPattern

locker

private static final KeyLocker<String> locker

A locker used to synchronize class loader initialization per coprocessor jar file

parentDirLockSet

static final HashSet<String> parentDirLockSet

A set used to synchronized parent path clean up. Generally, there should be only one parent path, but using a set so that we can support more.

Constructor Detail

CoprocessorClassLoader

private CoprocessorClassLoader(ClassLoader parent)

Creates a JarClassLoader that loads classes from the given paths.

Method Detail

init

private void init(org.apache.hadoop.fs.Path pathPattern,
                  String pathPrefix,
                  org.apache.hadoop.conf.Configuration conf)
           throws IOException

Throws:

IOException

getIfCached

public static CoprocessorClassLoader getIfCached(org.apache.hadoop.fs.Path path)

getAllCached

public static Collection<? extends ClassLoader> getAllCached()

clearCache

public static void clearCache()

getClassLoader

public static CoprocessorClassLoader getClassLoader(org.apache.hadoop.fs.Path path,
                                                    ClassLoader parent,
                                                    String pathPrefix,
                                                    org.apache.hadoop.conf.Configuration conf)
                                             throws IOException

Get a CoprocessorClassLoader for a coprocessor jar path from cache. If not in cache, create one.

Parameters:

path - the path to the coprocessor jar file to load classes from

parent - the parent class loader for exempted classes

pathPrefix - a prefix used in temp path name to store the jar file locally

conf - the configuration used to create the class loader, if needed

Returns:

a CoprocessorClassLoader for the coprocessor jar path

Throws:

IOException

loadClass

public Class<?> loadClass(String name)
                   throws ClassNotFoundException

Overrides:

loadClass in class ClassLoader

Throws:

ClassNotFoundException

loadClass

public Class<?> loadClass(String name,
                          String[] includedClassPrefixes)
                   throws ClassNotFoundException

Throws:

ClassNotFoundException

getResource

public URL getResource(String name)

Overrides:

getResource in class ClassLoader

isClassExempt

protected boolean isClassExempt(String name,
                                String[] includedClassPrefixes)

Determines whether the given class should be exempt from being loaded by this ClassLoader.

Parameters:

name - the name of the class to test.

Returns:

true if the class should *not* be loaded by this ClassLoader; false otherwise.

loadResourceUsingParentFirst

protected boolean loadResourceUsingParentFirst(String name)

Determines whether we should attempt to load the given resource using the parent first before attempting to load the resource using this ClassLoader.

Parameters:

name - the name of the resource to test.

Returns:

true if we should attempt to load the resource using the parent first; false if we should attempt to load the resource using this ClassLoader first.