org.apache.hadoop.hbase.regionserver

Class HRegionFileSystem

java.lang.Object

org.apache.hadoop.hbase.regionserver.HRegionFileSystem

Direct Known Subclasses:

HRegionWALFileSystem

@InterfaceAudience.Private
public class HRegionFileSystem
extends Object

View to an on-disk Region. Provides the set of methods necessary to interact with the on-disk region data.

Field Summary

Fields

Modifier and Type	Field and Description
private int	baseSleepBeforeRetries
(package private) org.apache.hadoop.conf.Configuration	conf
private static int	DEFAULT_BASE_SLEEP_BEFORE_RETRIES
private static int	DEFAULT_HDFS_CLIENT_RETRIES_NUMBER
(package private) org.apache.hadoop.fs.FileSystem	fs
private int	hdfsClientRetriesNumber In order to handle NN connectivity hiccups, one need to retry non-idempotent operation at the client level.
private static org.slf4j.Logger	LOG
static String	REGION_INFO_FILE Name of the region info file that resides just under the region directory.
static String	REGION_MERGES_DIR Temporary subdirectory of the region directory used for merges.
static String	REGION_SPLITS_DIR Temporary subdirectory of the region directory used for splits.
(package private) static String	REGION_TEMP_DIR Temporary subdirectory of the region directory used for compaction output.
private org.apache.hadoop.fs.Path	regionDir
private RegionInfo	regionInfo
(package private) RegionInfo	regionInfoForFs
private org.apache.hadoop.fs.Path	tableDir

Constructor Summary

Constructors

Constructor and Description
HRegionFileSystem(org.apache.hadoop.conf.Configuration conf, org.apache.hadoop.fs.FileSystem fs, org.apache.hadoop.fs.Path tableDir, RegionInfo regionInfo) Create a view to the on-disk region

Method Summary

Modifier and Type	Method and Description
(package private) Pair<org.apache.hadoop.fs.Path,org.apache.hadoop.fs.Path>	bulkLoadStoreFile(String familyName, org.apache.hadoop.fs.Path srcPath, long seqNum) Bulk load: Add a specified store file to the specified family.
(package private) void	checkRegionInfoOnFilesystem() Write out an info file under the stored region directory.
(package private) void	cleanupAnySplitDetritus() Clean up any split detritus that may have been left around from previous split attempts.
(package private) void	cleanupDaughterRegion(RegionInfo regionInfo) Remove daughter region
void	cleanupMergedRegion(RegionInfo mergedRegion) Remove merged region
(package private) void	cleanupMergesDir() Clean up any merge detritus that may have been left around from previous merge attempts.
(package private) void	cleanupSplitsDir() Clean up any split detritus that may have been left around from previous split attempts.
(package private) void	cleanupTempDir() Clean up any temp detritus that may have been left around from previous operation attempts.
org.apache.hadoop.fs.Path	commitDaughterRegion(RegionInfo regionInfo) Commit a daughter region, moving it from the split temporary directory to the proper location in the filesystem.
void	commitMergedRegion(RegionInfo mergedRegionInfo) Commit a merged region, moving it from the merges temporary directory to the proper location in the filesystem.
(package private) org.apache.hadoop.fs.Path	commitStoreFile(org.apache.hadoop.fs.Path buildPath, org.apache.hadoop.fs.Path dstPath)
org.apache.hadoop.fs.Path	commitStoreFile(String familyName, org.apache.hadoop.fs.Path buildPath) Move the file from a build/temp location to the main family store directory.
(package private) boolean	createDir(org.apache.hadoop.fs.Path dir) Creates a directory.
private static boolean	createDirOnFileSystem(org.apache.hadoop.fs.FileSystem fs, org.apache.hadoop.conf.Configuration conf, org.apache.hadoop.fs.Path dir) Creates a directory for a filesystem and configuration object.
void	createMergesDir() Create the region merges directory, a temporary directory to accumulate merges in.
static HRegionFileSystem	createRegionOnFileSystem(org.apache.hadoop.conf.Configuration conf, org.apache.hadoop.fs.FileSystem fs, org.apache.hadoop.fs.Path tableDir, RegionInfo regionInfo) Create a new Region on file-system.
void	createSplitsDir(RegionInfo daughterA, RegionInfo daughterB) Create the region splits directory.
(package private) org.apache.hadoop.fs.Path	createStoreDir(String familyName) Create the store directory for the specified family name
org.apache.hadoop.fs.Path	createTempName() Generate a unique temporary Path.
org.apache.hadoop.fs.Path	createTempName(String suffix) Generate a unique temporary Path.
(package private) boolean	deleteDir(org.apache.hadoop.fs.Path dir) Deletes a directory.
void	deleteFamily(String familyName) Remove the region family from disk, archiving the store files.
static void	deleteRegionFromFileSystem(org.apache.hadoop.conf.Configuration conf, org.apache.hadoop.fs.FileSystem fs, org.apache.hadoop.fs.Path tableDir, RegionInfo regionInfo) Remove the region from the table directory, archiving the region's hfiles.
private static String	generateUniqueName(String suffix) Generate a unique file name, used by createTempName() and commitStoreFile()
Collection<String>	getFamilies()
org.apache.hadoop.fs.FileSystem	getFileSystem()
org.apache.hadoop.fs.Path	getMergesDir()
(package private) org.apache.hadoop.fs.Path	getMergesDir(RegionInfo hri)
org.apache.hadoop.fs.Path	getRegionDir()
RegionInfo	getRegionInfo()
private static byte[]	getRegionInfoFileContent(RegionInfo hri)
RegionInfo	getRegionInfoForFS()
(package private) org.apache.hadoop.fs.Path	getSplitsDir()
org.apache.hadoop.fs.Path	getSplitsDir(RegionInfo hri)
String	getStoragePolicyName(String familyName) Get the storage policy of the directory of CF.
org.apache.hadoop.fs.Path	getStoreDir(String familyName) Returns the directory path of the specified family
(package private) StoreFileInfo	getStoreFileInfo(String familyName, String fileName) Return the store file information of the specified family/file.
(package private) org.apache.hadoop.fs.Path	getStoreFilePath(String familyName, String fileName) Return Qualified Path of the specified family/file
Collection<StoreFileInfo>	getStoreFiles(byte[] familyName) Returns the store files available for the family.
Collection<StoreFileInfo>	getStoreFiles(String familyName)
Collection<StoreFileInfo>	getStoreFiles(String familyName, boolean validate) Returns the store files available for the family.
static List<org.apache.hadoop.fs.LocatedFileStatus>	getStoreFilesLocatedStatus(HRegionFileSystem regionfs, String familyName, boolean validate) Returns the store files' LocatedFileStatus which available for the family.
org.apache.hadoop.fs.Path	getTableDir()
(package private) org.apache.hadoop.fs.Path	getTempDir()
boolean	hasReferences(String familyName) Returns true if the specified family has reference files
boolean	hasReferences(TableDescriptor htd) Check whether region has Reference file
static RegionInfo	loadRegionInfoFileContent(org.apache.hadoop.fs.FileSystem fs, org.apache.hadoop.fs.Path regionDir) Create a RegionInfo from the serialized version on-disk.
(package private) void	logFileSystemState(org.slf4j.Logger LOG) Log the current state of the region
org.apache.hadoop.fs.Path	mergeStoreFile(RegionInfo mergedRegion, String familyName, HStoreFile f, org.apache.hadoop.fs.Path mergedDir) Write out a merge reference under the given merges directory.
(package private) static boolean	mkdirs(org.apache.hadoop.fs.FileSystem fs, org.apache.hadoop.conf.Configuration conf, org.apache.hadoop.fs.Path dir)
static HRegionFileSystem	openRegionFromFileSystem(org.apache.hadoop.conf.Configuration conf, org.apache.hadoop.fs.FileSystem fs, org.apache.hadoop.fs.Path tableDir, RegionInfo regionInfo, boolean readOnly) Open Region from file-system.
private org.apache.hadoop.fs.Path	preCommitStoreFile(String familyName, org.apache.hadoop.fs.Path buildPath, long seqNum, boolean generateNewName) Generate the filename in the main family store directory for moving the file from a build/temp location.
void	removeStoreFile(String familyName, org.apache.hadoop.fs.Path filePath) Archives the specified store file from the specified family.
void	removeStoreFiles(String familyName, Collection<HStoreFile> storeFiles) Closes and archives the specified store files from the specified family.
(package private) boolean	rename(org.apache.hadoop.fs.Path srcpath, org.apache.hadoop.fs.Path dstPath) Renames a directory.
void	setStoragePolicy(String familyName, String policyName) Set the directory of CF to the specified storage policy.
private void	sleepBeforeRetry(String msg, int sleepMultiplier) sleeping logic; handles the interrupt exception.
private static void	sleepBeforeRetry(String msg, int sleepMultiplier, int baseSleepBeforeRetries, int hdfsClientRetriesNumber) sleeping logic for static methods; handles the interrupt exception.
org.apache.hadoop.fs.Path	splitStoreFile(RegionInfo hri, String familyName, HStoreFile f, byte[] splitRow, boolean top, RegionSplitPolicy splitPolicy) Write out a split reference.
private static void	writeRegionInfoFileContent(org.apache.hadoop.conf.Configuration conf, org.apache.hadoop.fs.FileSystem fs, org.apache.hadoop.fs.Path regionInfoFile, byte[] content) Write the .regioninfo file on-disk.
private void	writeRegionInfoOnFilesystem(boolean useTempDir) Write out an info file under the region directory.
private void	writeRegionInfoOnFilesystem(byte[] regionInfoContent, boolean useTempDir) Write out an info file under the region directory.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

LOG

private static final org.slf4j.Logger LOG

REGION_INFO_FILE

public static final String REGION_INFO_FILE

Name of the region info file that resides just under the region directory.

See Also:

Constant Field Values

REGION_MERGES_DIR

public static final String REGION_MERGES_DIR

Temporary subdirectory of the region directory used for merges.

See Also:

Constant Field Values

REGION_SPLITS_DIR

public static final String REGION_SPLITS_DIR

Temporary subdirectory of the region directory used for splits.

See Also:

Constant Field Values

REGION_TEMP_DIR

static final String REGION_TEMP_DIR

Temporary subdirectory of the region directory used for compaction output.

See Also:

Constant Field Values

regionInfo

private final RegionInfo regionInfo

regionInfoForFs

final RegionInfo regionInfoForFs

conf

final org.apache.hadoop.conf.Configuration conf

tableDir

private final org.apache.hadoop.fs.Path tableDir

fs

final org.apache.hadoop.fs.FileSystem fs

regionDir

private final org.apache.hadoop.fs.Path regionDir

hdfsClientRetriesNumber

private final int hdfsClientRetriesNumber

In order to handle NN connectivity hiccups, one need to retry non-idempotent operation at the client level.

baseSleepBeforeRetries

private final int baseSleepBeforeRetries

DEFAULT_HDFS_CLIENT_RETRIES_NUMBER

private static final int DEFAULT_HDFS_CLIENT_RETRIES_NUMBER

See Also:

Constant Field Values

DEFAULT_BASE_SLEEP_BEFORE_RETRIES

private static final int DEFAULT_BASE_SLEEP_BEFORE_RETRIES

See Also:

Constant Field Values

Constructor Detail

HRegionFileSystem

HRegionFileSystem(org.apache.hadoop.conf.Configuration conf,
                  org.apache.hadoop.fs.FileSystem fs,
                  org.apache.hadoop.fs.Path tableDir,
                  RegionInfo regionInfo)

Create a view to the on-disk region

Parameters:

conf - the Configuration to use

fs - FileSystem that contains the region

tableDir - Path to where the table is being stored

regionInfo - RegionInfo for region

Method Detail

getFileSystem

public org.apache.hadoop.fs.FileSystem getFileSystem()

Returns:

the underlying FileSystem

getRegionInfo

public RegionInfo getRegionInfo()

Returns:

the RegionInfo that describe this on-disk region view

getRegionInfoForFS

public RegionInfo getRegionInfoForFS()

getTableDir

public org.apache.hadoop.fs.Path getTableDir()

Returns:

Path to the region's root directory.

getRegionDir

public org.apache.hadoop.fs.Path getRegionDir()

Returns:

Path to the region directory.

getTempDir

org.apache.hadoop.fs.Path getTempDir()

Returns:

Path to the region's temp directory, used for file creations

cleanupTempDir

void cleanupTempDir()
             throws IOException

Clean up any temp detritus that may have been left around from previous operation attempts.

Throws:

IOException

getStoreDir

public org.apache.hadoop.fs.Path getStoreDir(String familyName)

Returns the directory path of the specified family

Parameters:

familyName - Column Family Name

Returns:

Path to the directory of the specified family

createStoreDir

org.apache.hadoop.fs.Path createStoreDir(String familyName)
                                  throws IOException

Create the store directory for the specified family name

Parameters:

familyName - Column Family Name

Returns:

Path to the directory of the specified family

Throws:

IOException - if the directory creation fails.

setStoragePolicy

public void setStoragePolicy(String familyName,
                             String policyName)

Set the directory of CF to the specified storage policy.
"LAZY_PERSIST", "ALL_SSD", "ONE_SSD", "HOT", "WARM", "COLD"

See HdfsConstants for more details.

Parameters:

familyName - The name of column family.

policyName - The name of the storage policy: 'HOT', 'COLD', etc. See see hadoop 2.6+ org.apache.hadoop.hdfs.protocol.HdfsConstants for possible list e.g 'COLD', 'WARM', 'HOT', 'ONE_SSD', 'ALL_SSD', 'LAZY_PERSIST'.

getStoragePolicyName

@Nullable
public String getStoragePolicyName(String familyName)

Get the storage policy of the directory of CF.

Parameters:

familyName - The name of column family.

Returns:

Storage policy name, or null if not using HFileSystem or exception thrown when trying to get policy

getStoreFiles

public Collection<StoreFileInfo> getStoreFiles(byte[] familyName)
                                        throws IOException

Returns the store files available for the family. This methods performs the filtering based on the valid store files.

Parameters:

familyName - Column Family Name

Returns:

a set of StoreFileInfo for the specified family.

Throws:

IOException

getStoreFiles

public Collection<StoreFileInfo> getStoreFiles(String familyName)
                                        throws IOException

Throws:

IOException

getStoreFiles

public Collection<StoreFileInfo> getStoreFiles(String familyName,
                                               boolean validate)
                                        throws IOException

Returns the store files available for the family. This methods performs the filtering based on the valid store files.

Parameters:

familyName - Column Family Name

Returns:

a set of StoreFileInfo for the specified family.

Throws:

IOException

getStoreFilesLocatedStatus

public static List<org.apache.hadoop.fs.LocatedFileStatus> getStoreFilesLocatedStatus(HRegionFileSystem regionfs,
                                                                                      String familyName,
                                                                                      boolean validate)
                                                                               throws IOException

Returns the store files' LocatedFileStatus which available for the family. This methods performs the filtering based on the valid store files.

Parameters:

familyName - Column Family Name

Returns:

a list of store files' LocatedFileStatus for the specified family.

Throws:

IOException

getStoreFilePath

org.apache.hadoop.fs.Path getStoreFilePath(String familyName,
                                           String fileName)

Return Qualified Path of the specified family/file

Parameters:

familyName - Column Family Name

fileName - File Name

Returns:

The qualified Path for the specified family/file

getStoreFileInfo

StoreFileInfo getStoreFileInfo(String familyName,
                               String fileName)
                        throws IOException

Return the store file information of the specified family/file.

Parameters:

familyName - Column Family Name

fileName - File Name

Returns:

The StoreFileInfo for the specified family/file

Throws:

IOException

hasReferences

public boolean hasReferences(String familyName)
                      throws IOException

Returns true if the specified family has reference files

Parameters:

familyName - Column Family Name

Returns:

true if family contains reference files

Throws:

IOException

hasReferences

public boolean hasReferences(TableDescriptor htd)
                      throws IOException

Check whether region has Reference file

Parameters:

htd - table desciptor of the region

Returns:

true if region has reference file

Throws:

IOException

getFamilies

public Collection<String> getFamilies()
                               throws IOException

Returns:

the set of families present on disk

Throws:

IOException

deleteFamily

public void deleteFamily(String familyName)
                  throws IOException

Remove the region family from disk, archiving the store files.

Parameters:

familyName - Column Family Name

Throws:

IOException - if an error occours during the archiving

generateUniqueName

private static String generateUniqueName(String suffix)

Generate a unique file name, used by createTempName() and commitStoreFile()

Parameters:

suffix - extra information to append to the generated name

Returns:

Unique file name

createTempName

public org.apache.hadoop.fs.Path createTempName()

Generate a unique temporary Path. Used in conjuction with commitStoreFile() to get a safer file creation. Path file = fs.createTempName(); ...StoreFile.Writer(file)... fs.commitStoreFile("family", file);

Returns:

Unique Path of the temporary file

createTempName

public org.apache.hadoop.fs.Path createTempName(String suffix)

Generate a unique temporary Path. Used in conjuction with commitStoreFile() to get a safer file creation. Path file = fs.createTempName(); ...StoreFile.Writer(file)... fs.commitStoreFile("family", file);

Parameters:

suffix - extra information to append to the generated name

Returns:

Unique Path of the temporary file

commitStoreFile

public org.apache.hadoop.fs.Path commitStoreFile(String familyName,
                                                 org.apache.hadoop.fs.Path buildPath)
                                          throws IOException

Move the file from a build/temp location to the main family store directory.

Parameters:

familyName - Family that will gain the file

buildPath - Path to the file to commit.

Returns:

The new Path of the committed file

Throws:

IOException

preCommitStoreFile

private org.apache.hadoop.fs.Path preCommitStoreFile(String familyName,
                                                     org.apache.hadoop.fs.Path buildPath,
                                                     long seqNum,
                                                     boolean generateNewName)
                                              throws IOException

Generate the filename in the main family store directory for moving the file from a build/temp location.

Parameters:

familyName - Family that will gain the file

buildPath - Path to the file to commit.

seqNum - Sequence Number to append to the file name (less then 0 if no sequence number)

generateNewName - False if you want to keep the buildPath name

Returns:

The new Path of the to be committed file

Throws:

IOException

commitStoreFile

org.apache.hadoop.fs.Path commitStoreFile(org.apache.hadoop.fs.Path buildPath,
                                          org.apache.hadoop.fs.Path dstPath)
                                   throws IOException

Throws:

IOException

removeStoreFile

public void removeStoreFile(String familyName,
                            org.apache.hadoop.fs.Path filePath)
                     throws IOException

Archives the specified store file from the specified family.

Parameters:

familyName - Family that contains the store files

filePath - Path to the store file to remove

Throws:

IOException - if the archiving fails

removeStoreFiles

public void removeStoreFiles(String familyName,
                             Collection<HStoreFile> storeFiles)
                      throws IOException

Closes and archives the specified store files from the specified family.

Parameters:

familyName - Family that contains the store files

storeFiles - set of store files to remove

Throws:

IOException - if the archiving fails

bulkLoadStoreFile

Pair<org.apache.hadoop.fs.Path,org.apache.hadoop.fs.Path> bulkLoadStoreFile(String familyName,
                                                                            org.apache.hadoop.fs.Path srcPath,
                                                                            long seqNum)
                                                                     throws IOException

Bulk load: Add a specified store file to the specified family. If the source file is on the same different file-system is moved from the source location to the destination location, otherwise is copied over.

Parameters:

familyName - Family that will gain the file

srcPath - Path to the file to import

seqNum - Bulk Load sequence number

Returns:

The destination Path of the bulk loaded file

Throws:

IOException

getSplitsDir

org.apache.hadoop.fs.Path getSplitsDir()

Returns:

Path to the temp directory used during split operations

getSplitsDir

public org.apache.hadoop.fs.Path getSplitsDir(RegionInfo hri)

cleanupSplitsDir

void cleanupSplitsDir()
               throws IOException

Clean up any split detritus that may have been left around from previous split attempts.

Throws:

IOException

cleanupAnySplitDetritus

void cleanupAnySplitDetritus()
                      throws IOException

Clean up any split detritus that may have been left around from previous split attempts. Call this method on initial region deploy.

Throws:

IOException

cleanupDaughterRegion

void cleanupDaughterRegion(RegionInfo regionInfo)
                    throws IOException

Remove daughter region

Parameters:

regionInfo - daughter RegionInfo

Throws:

IOException

commitDaughterRegion

public org.apache.hadoop.fs.Path commitDaughterRegion(RegionInfo regionInfo)
                                               throws IOException

Commit a daughter region, moving it from the split temporary directory to the proper location in the filesystem.

Parameters:

regionInfo - daughter RegionInfo

Throws:

IOException

createSplitsDir

public void createSplitsDir(RegionInfo daughterA,
                            RegionInfo daughterB)
                     throws IOException

Create the region splits directory.

Throws:

IOException

splitStoreFile

public org.apache.hadoop.fs.Path splitStoreFile(RegionInfo hri,
                                                String familyName,
                                                HStoreFile f,
                                                byte[] splitRow,
                                                boolean top,
                                                RegionSplitPolicy splitPolicy)
                                         throws IOException

Write out a split reference. Package local so it doesnt leak out of regionserver.

Parameters:

hri - RegionInfo of the destination

familyName - Column Family Name

f - File to split.

splitRow - Split Row

top - True if we are referring to the top half of the hfile.

splitPolicy - A split policy instance; be careful! May not be full populated; e.g. if this method is invoked on the Master side, then the RegionSplitPolicy will NOT have a reference to a Region.

Returns:

Path to created reference.

Throws:

IOException

getMergesDir

public org.apache.hadoop.fs.Path getMergesDir()

Returns:

Path to the temp directory used during merge operations

getMergesDir

org.apache.hadoop.fs.Path getMergesDir(RegionInfo hri)

cleanupMergesDir

void cleanupMergesDir()
               throws IOException

Clean up any merge detritus that may have been left around from previous merge attempts.

Throws:

IOException

cleanupMergedRegion

public void cleanupMergedRegion(RegionInfo mergedRegion)
                         throws IOException

Remove merged region

Parameters:

mergedRegion - RegionInfo

Throws:

IOException

mkdirs

static boolean mkdirs(org.apache.hadoop.fs.FileSystem fs,
                      org.apache.hadoop.conf.Configuration conf,
                      org.apache.hadoop.fs.Path dir)
               throws IOException

Throws:

IOException

createMergesDir

public void createMergesDir()
                     throws IOException

Create the region merges directory, a temporary directory to accumulate merges in.

Throws:

IOException - If merges dir already exists or we fail to create it.

See Also:

cleanupMergesDir()

mergeStoreFile

public org.apache.hadoop.fs.Path mergeStoreFile(RegionInfo mergedRegion,
                                                String familyName,
                                                HStoreFile f,
                                                org.apache.hadoop.fs.Path mergedDir)
                                         throws IOException

Write out a merge reference under the given merges directory. Package local so it doesnt leak out of regionserver.

Parameters:

mergedRegion - RegionInfo of the merged region

familyName - Column Family Name

f - File to create reference.

mergedDir -

Returns:

Path to created reference.

Throws:

IOException

commitMergedRegion

public void commitMergedRegion(RegionInfo mergedRegionInfo)
                        throws IOException

Commit a merged region, moving it from the merges temporary directory to the proper location in the filesystem.

Parameters:

mergedRegionInfo - merged region RegionInfo

Throws:

IOException

logFileSystemState

void logFileSystemState(org.slf4j.Logger LOG)
                 throws IOException

Log the current state of the region

Parameters:

LOG - log to output information

Throws:

IOException - if an unexpected exception occurs

getRegionInfoFileContent

private static byte[] getRegionInfoFileContent(RegionInfo hri)
                                        throws IOException

Parameters:

hri -

Returns:

Content of the file we write out to the filesystem under a region

Throws:

IOException

loadRegionInfoFileContent

public static RegionInfo loadRegionInfoFileContent(org.apache.hadoop.fs.FileSystem fs,
                                                   org.apache.hadoop.fs.Path regionDir)
                                            throws IOException

Create a RegionInfo from the serialized version on-disk.

Parameters:

fs - FileSystem that contains the Region Info file

regionDir - Path to the Region Directory that contains the Info file

Returns:

An RegionInfo instance gotten from the Region Info file.

Throws:

IOException - if an error occurred during file open/read operation.

writeRegionInfoFileContent

private static void writeRegionInfoFileContent(org.apache.hadoop.conf.Configuration conf,
                                               org.apache.hadoop.fs.FileSystem fs,
                                               org.apache.hadoop.fs.Path regionInfoFile,
                                               byte[] content)
                                        throws IOException

Write the .regioninfo file on-disk.

Overwrites if exists already.

Throws:

IOException

checkRegionInfoOnFilesystem

void checkRegionInfoOnFilesystem()
                          throws IOException

Write out an info file under the stored region directory. Useful recovering mangled regions. If the regionInfo already exists on-disk, then we fast exit.

Throws:

IOException

writeRegionInfoOnFilesystem

private void writeRegionInfoOnFilesystem(boolean useTempDir)
                                  throws IOException

Write out an info file under the region directory. Useful recovering mangled regions.

Parameters:

useTempDir - indicate whether or not using the region .tmp dir for a safer file creation.

Throws:

IOException

writeRegionInfoOnFilesystem

private void writeRegionInfoOnFilesystem(byte[] regionInfoContent,
                                         boolean useTempDir)
                                  throws IOException

Write out an info file under the region directory. Useful recovering mangled regions.

Parameters:

regionInfoContent - serialized version of the RegionInfo

useTempDir - indicate whether or not using the region .tmp dir for a safer file creation.

Throws:

IOException

createRegionOnFileSystem

public static HRegionFileSystem createRegionOnFileSystem(org.apache.hadoop.conf.Configuration conf,
                                                         org.apache.hadoop.fs.FileSystem fs,
                                                         org.apache.hadoop.fs.Path tableDir,
                                                         RegionInfo regionInfo)
                                                  throws IOException

Create a new Region on file-system.

Parameters:

conf - the Configuration to use

fs - FileSystem from which to add the region

tableDir - Path to where the table is being stored

regionInfo - RegionInfo for region to be added

Throws:

IOException - if the region creation fails due to a FileSystem exception.

openRegionFromFileSystem

public static HRegionFileSystem openRegionFromFileSystem(org.apache.hadoop.conf.Configuration conf,
                                                         org.apache.hadoop.fs.FileSystem fs,
                                                         org.apache.hadoop.fs.Path tableDir,
                                                         RegionInfo regionInfo,
                                                         boolean readOnly)
                                                  throws IOException

Open Region from file-system.

Parameters:

conf - the Configuration to use

fs - FileSystem from which to add the region

tableDir - Path to where the table is being stored

regionInfo - RegionInfo for region to be added

readOnly - True if you don't want to edit the region data

Throws:

IOException - if the region creation fails due to a FileSystem exception.

deleteRegionFromFileSystem

public static void deleteRegionFromFileSystem(org.apache.hadoop.conf.Configuration conf,
                                              org.apache.hadoop.fs.FileSystem fs,
                                              org.apache.hadoop.fs.Path tableDir,
                                              RegionInfo regionInfo)
                                       throws IOException

Remove the region from the table directory, archiving the region's hfiles.

Parameters:

conf - the Configuration to use

fs - FileSystem from which to remove the region

tableDir - Path to where the table is being stored

regionInfo - RegionInfo for region to be deleted

Throws:

IOException - if the request cannot be completed

createDir

boolean createDir(org.apache.hadoop.fs.Path dir)
           throws IOException

Creates a directory. Assumes the user has already checked for this directory existence.

Parameters:

dir -

Returns:

the result of fs.mkdirs(). In case underlying fs throws an IOException, it checks whether the directory exists or not, and returns true if it exists.

Throws:

IOException

rename

boolean rename(org.apache.hadoop.fs.Path srcpath,
               org.apache.hadoop.fs.Path dstPath)
        throws IOException

Renames a directory. Assumes the user has already checked for this directory existence.

Parameters:

srcpath -

dstPath -

Returns:

true if rename is successful.

Throws:

IOException

deleteDir

boolean deleteDir(org.apache.hadoop.fs.Path dir)
           throws IOException

Deletes a directory. Assumes the user has already checked for this directory existence.

Parameters:

dir -

Returns:

true if the directory is deleted.

Throws:

IOException

sleepBeforeRetry

private void sleepBeforeRetry(String msg,
                              int sleepMultiplier)
                       throws InterruptedException

sleeping logic; handles the interrupt exception.

Throws:

InterruptedException

createDirOnFileSystem

private static boolean createDirOnFileSystem(org.apache.hadoop.fs.FileSystem fs,
                                             org.apache.hadoop.conf.Configuration conf,
                                             org.apache.hadoop.fs.Path dir)
                                      throws IOException

Creates a directory for a filesystem and configuration object. Assumes the user has already checked for this directory existence.

Parameters:

fs -

conf -

dir -

Returns:

the result of fs.mkdirs(). In case underlying fs throws an IOException, it checks whether the directory exists or not, and returns true if it exists.

Throws:

IOException

sleepBeforeRetry

private static void sleepBeforeRetry(String msg,
                                     int sleepMultiplier,
                                     int baseSleepBeforeRetries,
                                     int hdfsClientRetriesNumber)
                              throws InterruptedException

sleeping logic for static methods; handles the interrupt exception. Keeping a static version for this to avoid re-looking for the integer values.

Throws:

InterruptedException