@InterfaceAudience.Private public class DynamicClassLoader extends ClassLoaderBase
The configured folder can be a HDFS path. In this case, the jar files under that folder will be copied to local at first under ${hbase.local.dir}/jars/. The local copy will be updated if the remote copy is updated, according to its last modified timestamp.
We can't unload a class already loaded. So we will use the existing jar files we already know to load any class which can't be loaded using the parent class loader. If we still can't load the class from the existing jar files, we will check if any new jar file is added, if so, we will load the new jar file and try to load the class again. If still failed, a class not found exception will be thrown.
Be careful in uploading new jar files and make sure all classes are consistent, otherwise, we may not be able to load your classes properly.
| Modifier and Type | Field and Description |
|---|---|
private static String |
DYNAMIC_JARS_DIR |
private static String |
DYNAMIC_JARS_DIR_KEY |
private static String |
DYNAMIC_JARS_OPTIONAL_CONF_KEY |
private static boolean |
DYNAMIC_JARS_OPTIONAL_DEFAULT |
private HashMap<String,Long> |
jarModifiedTime |
private File |
localDir |
private static org.apache.commons.logging.Log |
LOG |
private org.apache.hadoop.fs.Path |
remoteDir |
private org.apache.hadoop.fs.FileSystem |
remoteDirFs |
private boolean |
useDynamicJars |
DEFAULT_LOCAL_DIR, LOCAL_DIR_KEY, parent| Constructor and Description |
|---|
DynamicClassLoader(org.apache.hadoop.conf.Configuration conf,
ClassLoader parent)
Creates a DynamicClassLoader that can load classes dynamically
from jar files under a specific folder.
|
| Modifier and Type | Method and Description |
|---|---|
private void |
initTempDir(org.apache.hadoop.conf.Configuration conf) |
Class<?> |
loadClass(String name) |
private void |
loadNewJars() |
private Class<?> |
tryRefreshClass(String name) |
getClassLoadingLockaddURL, close, definePackage, findClass, findResource, findResources, getPermissions, getResourceAsStream, getURLs, newInstance, newInstancedefineClass, defineClassclearAssertionStatus, defineClass, defineClass, defineClass, defineClass, definePackage, findLibrary, findLoadedClass, findSystemClass, getPackage, getPackages, getParent, getResource, getResources, getSystemClassLoader, getSystemResource, getSystemResourceAsStream, getSystemResources, loadClass, registerAsParallelCapable, resolveClass, setClassAssertionStatus, setDefaultAssertionStatus, setPackageAssertionStatus, setSignersprivate static final org.apache.commons.logging.Log LOG
private static final String DYNAMIC_JARS_DIR
private static final String DYNAMIC_JARS_DIR_KEY
private static final String DYNAMIC_JARS_OPTIONAL_CONF_KEY
private static final boolean DYNAMIC_JARS_OPTIONAL_DEFAULT
private boolean useDynamicJars
private File localDir
private org.apache.hadoop.fs.FileSystem remoteDirFs
private org.apache.hadoop.fs.Path remoteDir
public DynamicClassLoader(org.apache.hadoop.conf.Configuration conf,
ClassLoader parent)
conf - the configuration for the cluster.parent - the parent ClassLoader to set.private void initTempDir(org.apache.hadoop.conf.Configuration conf)
public Class<?> loadClass(String name) throws ClassNotFoundException
loadClass in class ClassLoaderClassNotFoundExceptionprivate Class<?> tryRefreshClass(String name) throws ClassNotFoundException
ClassNotFoundExceptionprivate void loadNewJars()
Copyright © 2007–2019 The Apache Software Foundation. All rights reserved.