org.apache.hadoop.hbase.util

Class FSHDFSUtils

java.lang.Object

org.apache.hadoop.hbase.util.CommonFSUtils

org.apache.hadoop.hbase.util.FSUtils

org.apache.hadoop.hbase.util.FSHDFSUtils

@InterfaceAudience.Private
 @InterfaceStability.Evolving
public class FSHDFSUtils
extends FSUtils

Implementation for hdfs

Nested Class Summary

Nested classes/interfaces inherited from class org.apache.hadoop.hbase.util.FSUtils

FSUtils.BlackListDirFilter, FSUtils.DirFilter, FSUtils.FamilyDirFilter, FSUtils.FileFilter, FSUtils.HFileFilter, FSUtils.HFileLinkFilter, FSUtils.ProgressReporter, FSUtils.ReferenceFileFilter, FSUtils.RegionDirFilter, FSUtils.UserTableDirFilter

Nested classes/interfaces inherited from class org.apache.hadoop.hbase.util.CommonFSUtils

CommonFSUtils.StreamLacksCapabilityException

Field Summary

Fields

Modifier and Type	Field and Description
private static Class	dfsUtilClazz
private static Method	getNNAddressesMethod
private static org.slf4j.Logger	LOG

Fields inherited from class org.apache.hadoop.hbase.util.FSUtils

WINDOWS

Fields inherited from class org.apache.hadoop.hbase.util.CommonFSUtils

FULL_RWX_PERMISSIONS, HBASE_WAL_DIR, UNSAFE_STREAM_CAPABILITY_ENFORCE

Constructor Summary

Constructors

Constructor and Description
FSHDFSUtils()

Method Summary

Modifier and Type	Method and Description
(package private) void	checkIfCancelled(CancelableProgressable reporter)
(package private) boolean	checkIfTimedout(org.apache.hadoop.conf.Configuration conf, long recoveryTimeout, int nbAttempt, org.apache.hadoop.fs.Path p, long startWaiting)
private String	getLogMessageDetail(int nbAttempt, org.apache.hadoop.fs.Path p, long startWaiting)
private static Set<InetSocketAddress>	getNNAddresses(org.apache.hadoop.hdfs.DistributedFileSystem fs, org.apache.hadoop.conf.Configuration conf)
private boolean	isFileClosed(org.apache.hadoop.hdfs.DistributedFileSystem dfs, Method m, org.apache.hadoop.fs.Path p) Call HDFS-4525 isFileClosed if it is available.
static boolean	isSameHdfs(org.apache.hadoop.conf.Configuration conf, org.apache.hadoop.fs.FileSystem srcFs, org.apache.hadoop.fs.FileSystem desFs)
(package private) boolean	recoverDFSFileLease(org.apache.hadoop.hdfs.DistributedFileSystem dfs, org.apache.hadoop.fs.Path p, org.apache.hadoop.conf.Configuration conf, CancelableProgressable reporter)
void	recoverFileLease(org.apache.hadoop.fs.FileSystem fs, org.apache.hadoop.fs.Path p, org.apache.hadoop.conf.Configuration conf, CancelableProgressable reporter) Recover the lease from HDFS, retrying multiple times.
(package private) boolean	recoverLease(org.apache.hadoop.hdfs.DistributedFileSystem dfs, int nbAttempt, org.apache.hadoop.fs.Path p, long startWaiting) Try to recover the lease.

Methods inherited from class org.apache.hadoop.hbase.util.FSUtils

addToHDFSBlocksDistribution, checkAccess, checkClusterIdExists, checkDfsSafeMode, checkFileSystemAvailable, checkShortCircuitReadBufferSize, checkVersion, checkVersion, computeHDFSBlocksDistribution, copyFilesParallel, create, deleteRegionDir, filterFileStatuses, filterFileStatuses, getClusterId, getDFSHedgedReadMetrics, getFamilyDirs, getInstance, getLocalTableDirs, getReferenceFilePaths, getRegionDegreeLocalityMappingFromFS, getRegionDegreeLocalityMappingFromFS, getRegionDirFromRootDir, getRegionDirFromTableDir, getRegionDirs, getRegionReferenceFileCount, getTableDirs, getTableFragmentation, getTableFragmentation, getTableStoreFilePathMap, getTableStoreFilePathMap, getTableStoreFilePathMap, getTableStoreFilePathMap, getTableStoreFilePathMap, getTableStoreFilePathMap, getTotalTableFragmentation, getVersion, isDistributedFileSystem, isMatchingTail, listStatusWithStatusFilter, metaRegionExists, parseVersionFrom, setClusterId, setupShortCircuitRead, setVersion, setVersion, setVersion, toVersionByteArray, waitOnSafeMode

Methods inherited from class org.apache.hadoop.hbase.util.CommonFSUtils

create, createForWal, createForWal, delete, deleteDirectory, getCurrentFileSystem, getDefaultBlockSize, getDefaultBufferSize, getDefaultReplication, getFilePermissions, getNamespaceDir, getPath, getRootDir, getRootDirFileSystem, getTableDir, getTableName, getWALFileSystem, getWALRegionDir, getWALRootDir, getWALTableDir, getWrongWALRegionDir, hasCapability, isExists, isHDFS, isMatchingTail, isRecoveredEdits, isStartingWithPath, listLocatedStatus, listStatus, listStatus, logFileSystemState, removeWALRootPath, renameAndSetModifyTime, setFsDefault, setRootDir, setStoragePolicy, setStoragePolicy, setWALRootDir, validateRootPath

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

LOG

private static final org.slf4j.Logger LOG

dfsUtilClazz

private static Class dfsUtilClazz

getNNAddressesMethod

private static Method getNNAddressesMethod

Constructor Detail

FSHDFSUtils

public FSHDFSUtils()

Method Detail

getNNAddresses

private static Set<InetSocketAddress> getNNAddresses(org.apache.hadoop.hdfs.DistributedFileSystem fs,
                                                     org.apache.hadoop.conf.Configuration conf)

Parameters:

fs -

conf -

Returns:

A set containing all namenode addresses of fs

isSameHdfs

public static boolean isSameHdfs(org.apache.hadoop.conf.Configuration conf,
                                 org.apache.hadoop.fs.FileSystem srcFs,
                                 org.apache.hadoop.fs.FileSystem desFs)

Parameters:

conf - the Configuration of HBase

srcFs -

desFs -

Returns:

Whether srcFs and desFs are on same hdfs or not

recoverFileLease

public void recoverFileLease(org.apache.hadoop.fs.FileSystem fs,
                             org.apache.hadoop.fs.Path p,
                             org.apache.hadoop.conf.Configuration conf,
                             CancelableProgressable reporter)
                      throws IOException

Recover the lease from HDFS, retrying multiple times.

Specified by:

recoverFileLease in class FSUtils

Parameters:

fs - FileSystem handle

p - Path of file to recover lease

conf - Configuration handle

Throws:

IOException

recoverDFSFileLease

boolean recoverDFSFileLease(org.apache.hadoop.hdfs.DistributedFileSystem dfs,
                            org.apache.hadoop.fs.Path p,
                            org.apache.hadoop.conf.Configuration conf,
                            CancelableProgressable reporter)
                     throws IOException

Throws:

IOException

checkIfTimedout

boolean checkIfTimedout(org.apache.hadoop.conf.Configuration conf,
                        long recoveryTimeout,
                        int nbAttempt,
                        org.apache.hadoop.fs.Path p,
                        long startWaiting)

recoverLease

boolean recoverLease(org.apache.hadoop.hdfs.DistributedFileSystem dfs,
                     int nbAttempt,
                     org.apache.hadoop.fs.Path p,
                     long startWaiting)
              throws FileNotFoundException

Try to recover the lease.

Parameters:

dfs -

nbAttempt -

p -

startWaiting -

Returns:

True if dfs#recoverLease came by true.

Throws:

FileNotFoundException

getLogMessageDetail

private String getLogMessageDetail(int nbAttempt,
                                   org.apache.hadoop.fs.Path p,
                                   long startWaiting)

Parameters:

nbAttempt -

p -

startWaiting -

Returns:

Detail to append to any log message around lease recovering.

isFileClosed

private boolean isFileClosed(org.apache.hadoop.hdfs.DistributedFileSystem dfs,
                             Method m,
                             org.apache.hadoop.fs.Path p)

Call HDFS-4525 isFileClosed if it is available.

Parameters:

dfs -

m -

p -

Returns:

True if file is closed.

checkIfCancelled

void checkIfCancelled(CancelableProgressable reporter)
               throws InterruptedIOException

Throws:

InterruptedIOException