HFileSystem (Apache HBase 1.2.12 API)

java.lang.Object
- org.apache.hadoop.conf.Configured
- - org.apache.hadoop.fs.FileSystem
  - - org.apache.hadoop.fs.FilterFileSystem
    - - org.apache.hadoop.hbase.fs.HFileSystem

All Implemented Interfaces:

Closeable, AutoCloseable, org.apache.hadoop.conf.Configurable
```
public class HFileSystem
extends org.apache.hadoop.fs.FilterFileSystem
```
An encapsulation for the FileSystem object that hbase uses to access data. This class allows the flexibility of using separate filesystem objects for reading and writing hfiles and wals. In future, if we want to make wals be in a different filesystem, this is the place to make it happen.

Nested Class Summary

Nested Classes
Modifier and Type	Class and Description
`(package private) static interface`	`HFileSystem.ReorderBlocks` Interface to implement to add a specific reordering logic in hdfs.
`(package private) static class`	`HFileSystem.ReorderWALBlocks` We're putting at lowest priority the wal files blocks that are on the same datanode as the original regionserver which created these files.

Nested classes/interfaces inherited from class org.apache.hadoop.fs.FileSystem
org.apache.hadoop.fs.FileSystem.Statistics

Field Summary

Fields
Modifier and Type Field and Description

static org.apache.commons.logging.Log LOG

private org.apache.hadoop.fs.FileSystem noChecksumFs

private boolean useHBaseChecksum
- Fields inherited from class org.apache.hadoop.fs.FilterFileSystem
  fs, swapScheme
- Fields inherited from class org.apache.hadoop.fs.FileSystem
  DEFAULT_FS, FS_DEFAULT_NAME_KEY, SHUTDOWN_HOOK_PRIORITY, statistics

Fields
Modifier and Type	Field and Description
`static org.apache.commons.logging.Log`	`LOG`
`private org.apache.hadoop.fs.FileSystem`	`noChecksumFs`
`private boolean`	`useHBaseChecksum`

Constructor Summary

Constructors
Constructor and Description
`HFileSystem(org.apache.hadoop.conf.Configuration conf, boolean useHBaseChecksum)` Create a FileSystem object for HBase regionservers.
`HFileSystem(org.apache.hadoop.fs.FileSystem fs)` Wrap a FileSystem object within a HFileSystem.

Method Summary

Methods
Modifier and Type	Method and Description
`static boolean`	`addLocationsOrderInterceptor(org.apache.hadoop.conf.Configuration conf)`
`(package private) static boolean`	`addLocationsOrderInterceptor(org.apache.hadoop.conf.Configuration conf, HFileSystem.ReorderBlocks lrb)` Add an interceptor on the calls to the namenode#getBlockLocations from the DFSClient linked to this FileSystem.
`void`	`close()` Close this filesystem object
`org.apache.hadoop.fs.FSDataOutputStream`	`createNonRecursive(org.apache.hadoop.fs.Path f, boolean overwrite, int bufferSize, short replication, long blockSize, org.apache.hadoop.util.Progressable progress)` The org.apache.hadoop.fs.FilterFileSystem does not yet support createNonRecursive.
`private static org.apache.hadoop.hdfs.protocol.ClientProtocol`	`createReorderingProxy(org.apache.hadoop.hdfs.protocol.ClientProtocol cp, HFileSystem.ReorderBlocks lrb, org.apache.hadoop.conf.Configuration conf)`
`static org.apache.hadoop.fs.FileSystem`	`get(org.apache.hadoop.conf.Configuration conf)` Create a new HFileSystem object, similar to FileSystem.get().
`org.apache.hadoop.fs.FileSystem`	`getBackingFs()` Returns the underlying filesystem
`static org.apache.hadoop.fs.FileSystem`	`getLocalFs(org.apache.hadoop.conf.Configuration conf)` Wrap a LocalFileSystem within a HFileSystem.
`org.apache.hadoop.fs.FileSystem`	`getNoChecksumFs()` Returns the filesystem that is specially setup for doing reads from storage.
`private static org.apache.hadoop.fs.FileSystem`	`newInstanceFileSystem(org.apache.hadoop.conf.Configuration conf)` Returns a brand new instance of the FileSystem.
`boolean`	`useHBaseChecksum()` Are we verifying checksums in HBase?

Methods inherited from class org.apache.hadoop.fs.FilterFileSystem
append, canonicalizeUri, checkPath, completeLocalOutput, concat, copyFromLocalFile, copyFromLocalFile, copyFromLocalFile, copyToLocalFile, create, createNonRecursive, createSnapshot, createSymlink, delete, deleteSnapshot, getAclStatus, getCanonicalUri, getChildFileSystems, getConf, getDefaultBlockSize, getDefaultBlockSize, getDefaultReplication, getDefaultReplication, getFileBlockLocations, getFileChecksum, getFileChecksum, getFileLinkStatus, getFileStatus, getHomeDirectory, getInitialWorkingDirectory, getLinkTarget, getRawFileSystem, getServerDefaults, getServerDefaults, getStatus, getUri, getUsed, getWorkingDirectory, getXAttr, getXAttrs, getXAttrs, initialize, listCorruptFileBlocks, listLocatedStatus, listStatus, listXAttrs, makeQualified, mkdirs, modifyAclEntries, open, primitiveCreate, primitiveMkdir, removeAcl, removeAclEntries, removeDefaultAcl, removeXAttr, rename, renameSnapshot, resolveLink, resolvePath, setAcl, setOwner, setPermission, setReplication, setTimes, setVerifyChecksum, setWorkingDirectory, setWriteChecksum, setXAttr, setXAttr, startLocalOutput, supportsSymlinks

Methods inherited from class org.apache.hadoop.fs.FileSystem
addDelegationTokens, append, append, areSymlinksEnabled, cancelDeleteOnExit, clearStatistics, closeAll, closeAllForUGI, copyFromLocalFile, copyToLocalFile, copyToLocalFile, create, create, create, create, create, create, create, create, create, create, create, create, createNewFile, createNonRecursive, createSnapshot, delete, deleteOnExit, enableSymlinks, exists, fixRelativePart, get, get, getAllStatistics, getBlockSize, getCanonicalServiceName, getContentSummary, getDefaultPort, getDefaultUri, getDelegationToken, getFileBlockLocations, getFileSystemClass, getFSofPath, getLength, getLocal, getName, getNamed, getReplication, getScheme, getStatistics, getStatistics, getStatus, globStatus, globStatus, isDirectory, isFile, listFiles, listLocatedStatus, listStatus, listStatus, listStatus, mkdirs, mkdirs, moveFromLocalFile, moveFromLocalFile, moveToLocalFile, newInstance, newInstance, newInstance, newInstanceLocal, open, primitiveMkdir, printStatistics, processDeleteOnExit, rename, setDefaultUri, setDefaultUri

Methods inherited from class org.apache.hadoop.conf.Configured
setConf

Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

- Field Detail
  - LOG
```
public static final org.apache.commons.logging.Log LOG
```
  - noChecksumFs
```
private final org.apache.hadoop.fs.FileSystem noChecksumFs
```
  - useHBaseChecksum
```
private final boolean useHBaseChecksum
```
- Constructor Detail
  - HFileSystem
```
public HFileSystem(org.apache.hadoop.conf.Configuration conf,
           boolean useHBaseChecksum)
            throws IOException
```
    Create a FileSystem object for HBase regionservers.
    
    Parameters:
    conf - The configuration to be used for the filesystem
    useHBaseChecksum - if true, then use checksum verfication in hbase, otherwise delegate checksum verification to the FileSystem.
    
    Throws:
    
    IOException
  - HFileSystem
```
public HFileSystem(org.apache.hadoop.fs.FileSystem fs)
```
    Wrap a FileSystem object within a HFileSystem. The noChecksumFs and writefs are both set to be the same specified fs. Do not verify hbase-checksums while reading data from filesystem.
    
    Parameters:
    fs - Set the noChecksumFs and writeFs to this specified filesystem.
- Method Detail
  - getNoChecksumFs
```
public org.apache.hadoop.fs.FileSystem getNoChecksumFs()
```
    Returns the filesystem that is specially setup for doing reads from storage. This object avoids doing checksum verifications for reads.
    
    Returns:
    The FileSystem object that can be used to read data from files.
  - getBackingFs
```
public org.apache.hadoop.fs.FileSystem getBackingFs()
                                             throws IOException
```
    Returns the underlying filesystem
    
    Returns:
    The underlying FileSystem for this FilterFileSystem object.
    
    Throws:
    
    IOException
  - useHBaseChecksum
```
public boolean useHBaseChecksum()
```
    Are we verifying checksums in HBase?
    
    Returns:
    True, if hbase is configured to verify checksums, otherwise false.
  - close
```
public void close()
           throws IOException
```
    Close this filesystem object
    
    Specified by:
    
    close in interface Closeable
    
    Specified by:
    
    close in interface AutoCloseable
    
    Overrides:
    
    close in class org.apache.hadoop.fs.FilterFileSystem
    
    Throws:
    
    IOException
  - newInstanceFileSystem
```
private static org.apache.hadoop.fs.FileSystem newInstanceFileSystem(org.apache.hadoop.conf.Configuration conf)
                                                              throws IOException
```
    Returns a brand new instance of the FileSystem. It does not use the FileSystem.Cache. In newer versions of HDFS, we can directly invoke FileSystem.newInstance(Configuration).
    
    Parameters:
    conf - Configuration
    
    Returns:
    A new instance of the filesystem
    
    Throws:
    
    IOException
  - addLocationsOrderInterceptor
```
public static boolean addLocationsOrderInterceptor(org.apache.hadoop.conf.Configuration conf)
                                            throws IOException
```
    Throws:
    
    IOException
  - addLocationsOrderInterceptor
```
static boolean addLocationsOrderInterceptor(org.apache.hadoop.conf.Configuration conf,
                                   HFileSystem.ReorderBlocks lrb)
```
    Add an interceptor on the calls to the namenode#getBlockLocations from the DFSClient linked to this FileSystem. See HBASE-6435 for the background.
    There should be no reason, except testing, to create a specific ReorderBlocks.
    
    Returns:
    true if the interceptor was added, false otherwise.
  - createReorderingProxy
```
private static org.apache.hadoop.hdfs.protocol.ClientProtocol createReorderingProxy(org.apache.hadoop.hdfs.protocol.ClientProtocol cp,
                                                                   HFileSystem.ReorderBlocks lrb,
                                                                   org.apache.hadoop.conf.Configuration conf)
```
  - get
```
public static org.apache.hadoop.fs.FileSystem get(org.apache.hadoop.conf.Configuration conf)
                                           throws IOException
```
    Create a new HFileSystem object, similar to FileSystem.get(). This returns a filesystem object that avoids checksum verification in the filesystem for hfileblock-reads. For these blocks, checksum verification is done by HBase.
    
    Throws:
    
    IOException
  - getLocalFs
```
public static org.apache.hadoop.fs.FileSystem getLocalFs(org.apache.hadoop.conf.Configuration conf)
                                                  throws IOException
```
    Wrap a LocalFileSystem within a HFileSystem.
    
    Throws:
    
    IOException
  - createNonRecursive
```
public org.apache.hadoop.fs.FSDataOutputStream createNonRecursive(org.apache.hadoop.fs.Path f,
                                                         boolean overwrite,
                                                         int bufferSize,
                                                         short replication,
                                                         long blockSize,
                                                         org.apache.hadoop.util.Progressable progress)
                                                           throws IOException
```
    The org.apache.hadoop.fs.FilterFileSystem does not yet support createNonRecursive. This is a hadoop bug and when it is fixed in Hadoop, this definition will go away.
    
    Overrides:
    
    createNonRecursive in class org.apache.hadoop.fs.FileSystem
    
    Throws:
    
    IOException

Class HFileSystem

Nested Class Summary

Nested classes/interfaces inherited from class org.apache.hadoop.fs.FileSystem

Field Summary

Fields inherited from class org.apache.hadoop.fs.FilterFileSystem

Fields inherited from class org.apache.hadoop.fs.FileSystem

Constructor Summary

Method Summary

Methods inherited from class org.apache.hadoop.fs.FilterFileSystem

Methods inherited from class org.apache.hadoop.fs.FileSystem

Methods inherited from class org.apache.hadoop.conf.Configured

Methods inherited from class java.lang.Object

Field Detail

LOG

noChecksumFs

useHBaseChecksum

Constructor Detail

HFileSystem

HFileSystem

Method Detail

getNoChecksumFs

getBackingFs

useHBaseChecksum

close

newInstanceFileSystem

addLocationsOrderInterceptor

addLocationsOrderInterceptor

createReorderingProxy

get

getLocalFs

createNonRecursive